Lately, I've been thinking about the idea of creating a web app for storing and visualizing my cycle rides. Most of the popular activity trackers, allow exporting your activities as GPX files. We can use those files to import activity to the other service, for example to the one that we will build in a moment.
In this blog post, I would like to present my findings on how to store and visualize GPX tracks using Elixir/Phoenix, PostgreSQL and a little bit of JavaScript. The plan is to parse the GPX file and extract track data. Save it in PostgreSQL as a geometry type, which comes with PostGIS. Finally, visualize track using an interactive web map.
GitHub repo containing code used in this blogpost
GPX intro
GPX (GPS Exchange Format) is an XML data format, designed to share GPS data between software applications. You can find more info about the format on the official website.
GPX can be used to describe the following data:
- waypoints - individual points without relation to each other
- routes - an ordered list of points, representing a series of turns leading to a destination
- tracks - an ordered list of points, describing a path, for example, a raw output of GPS recording of single trip
Below we can examine an example GPX file. It contains tracks data, which were recorded during an activity.
<!-- my_activity.xml -->
<?xml version="1.0" encoding="UTF-8"?>
<gpx creator="StravaGPX" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.topografix.com/GPX/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd" version="1.1" xmlns="http://www.topografix.com/GPX/1/1">
<metadata>
<time>2020-02-02T10:37:13Z</time>
</metadata>
<trk> <!-- representation of a track -->
<name>Gdynia</name>
<trkseg> <!-- track segment -->
<trkpt lat="54.5198480" lon="18.5396990"> <!-- track point -->
<ele>10.2</ele> <!-- point elevation -->
<time>2020-02-02T10:37:13Z</time> <!-- time of the recording -->
</trkpt>
<trkpt lat="54.5198540" lon="18.5397300">
<ele>10.2</ele>
<time>2020-02-02T10:37:14Z</time>
</trkpt>
<!-- more track points -->
</trkseg>
</trk>
</gpx>
To parse GPX files in Elixir, I’ve created GpxEx package. It’s still work in progress but it supports parsing tracks. After reading a file, you can convert it to Elixir structs.
{:ok, gpx_doc} = File.read("./my_track.gpx")
{:ok, gpx} = GpxEx.parse(gpx_doc)
%GpxEx.Gpx{
tracks: [
%GpxEx.Track{
name: "Track's name",
segments: [
%GpxEx.TrackSegment{
points: [
%GpxEx.TrackPoint{
ele: 10.2,
lat: 54.519848,
lon: 18.539699,
time: "2020-02-02T10:37:13Z"
},
%GpxEx.TrackPoint{
ele: 10.2,
lat: 54.519854,
lon: 18.53973,
time: "2020-02-02T10:37:14Z"
}
]
}
]
}
]
}
PostGIS intro
What is PostGIS and why do we need it? PostgreSQL supports the XML data type natively. Why not use that? We could save the GPX file straight away and skip the whole parsing part and adding the extra extension.
Doing all of that, we would lose many benefits that are provided by PostGIS. PostGIS is a spatial database extension that adds support for geographic objects. After converting tracks to geo type and storing them in Postgres, we will be able to run location queries and spatial functions. For example, we will be able to:
- find all tracks near a certain location
- calculate track's distance
- convert a track to the format used by web maps (GeoJSON, TopoJSON, KML, etc)
Intial project setup
In this tutorial I'm using the following versions:
- Elixir 1.10
- Phoenix 1.4.14
- PostgreSQL 12.2
- PostGIS 3.0
Let's start by creating a fresh Phoenix project.
mix phx.new gpx_phoenix
cd gpx_phoenix
mix ecto.create
PostGIS in Phoenix framework
We need to add PostGIS support in Phoenix application. To do that, we can use geo and geo_postgis packages.
defp deps do
# other deps
{:geo, "~> 3.3"},
{:geo_postgis, "~> 3.3"}
end
First, we need to pass new PostGIS extensions to postgrex. We have to create new file, for example lib/gpx_phoenix/postgrex_extensions.ex. It has to be defined only once during compilation, hence it needs to be done outside of any module or function.
# lib/gpx_phoenix/postgrex_extensions.ex
Postgrex.Types.define(
GpxPhoenix.PostgresTypes,
[Geo.PostGIS.Extension] ++ Ecto.Adapters.Postgres.extensions(),
json: Jason
)
After defining the above types, we need to specify them in our Repo config.
# config/config.exs
config :gpx_phoenix, GpxPhoenix.Repo,
types: GpxPhoenix.PostgresTypes
The last step is to enable PostGIS in PostgreSQL.
defmodule GpxPhoenix.Repo.Migrations.EnablePostgisExtension do
use Ecto.Migration
def up do
execute "CREATE EXTENSION IF NOT EXISTS postgis"
end
def down do
execute "DROP EXTENSION IF EXISTS postgis"
end
end
Tracks context
In this section, we are creating Tracks context. We have to generate migration which will create tracks table with two columns. geom column will hold the geometry of each track. We can create it using the AddGeometryColumn function provided by PostGIS.
-- AddGeometryColumn(table_name, column_name, srid, type, dimension);
SELECT AddGeometryColumn('tracks', 'geom', 3857, 'MULTILINESTRINGZ', 3);
- The
sirdstands for spatial reference system identifier which defines the coordinate system. We are going to use Pseudo-Mercator (EPSG:3857) used for rendering most of the popular web maps. - The
typespecifies geometry type, eg 'MULTILINESTRINGZ', 'POLYGON', 'POINT'. - The last argument is the
dimension. We want to store 3 dimensions, x and y coordinates along with elevation (z).
We are defining geom as MULTILINESTRINGZ because it plays nicely with the way how GPX format. GPX track can contain multiple segments, and each segment contains multiple points. Each point has (lat, lon) coordinates and can also hold elevation.
A linestring is a path between locations, an ordered series of two or more points. A "Z" dimension adds height information to each point. A MultiLineStringZ is a collection of linestringZ
# priv/repo/migrations/20200316162637_create_tracks_table.exs
defmodule GpxPhoenix.Repo.Migrations.CreateTracksTable do
use Ecto.Migration
def up do
create table(:tracks) do
add(:name, :string)
timestamps()
end
execute("SELECT AddGeometryColumn('tracks', 'geom', 3857, 'MULTILINESTRINGZ', 3);")
end
def down do
drop table(:tracks)
end
end
In track’s schema, we have to use Geo.PostGIS.Geometry type which was added by geo_postigs package. We have to remember that we specified our geometry type as MULTILINESTRINGZ and the database will enforce that.
# lib/gpx_phoenix/tracks/track.ex
defmodule GpxPhoenix.Tracks.Track do
use Ecto.Schema
import Ecto.Changeset
schema "tracks" do
field(:name, :string)
field(:geom, Geo.PostGIS.Geometry)
timestamps()
end
@doc false
def changeset(track, attrs) do
track
|> cast(attrs, [:name, :geom])
|> validate_required([:name, :geom])
end
end
Tracks context with basic CRUD functions.
# lib/gpx_phoenix/tracks/tracks.ex
defmodule GpxPhoenix.Tracks do
@moduledoc """
The Tracks context.
"""
import Ecto.Query, warn: false
alias GpxPhoenix.Repo
alias GpxPhoenix.Tracks.Track
def get_track!(id), do: Repo.get!(Track, id)
def list_tracks, do: Repo.all(Track)
def create_track(attrs \\ %{}) do
%Track{}
|> Track.changeset(attrs)
|> Repo.insert()
end
def change_track(%Track{} = track), do: Track.changeset(track, %{})
end
Tracks importer
Now we can focus on track importer module. It will be responsible for parsing GPX and creating a new track record. We will parse GPX file using GpxEx package which we have to add to our dependencies.
# mix.exs
defp deps do
# other deps
{:gpx_ex, git: "git@github.com:caspg/gpx_ex.git", tag: "0.1.0"}
end
Before saving parsed GPX file to the database, we have to convert it to our geometry type, which is Geo.MultiLineStringZ. When creating Geo type, we have to use the same srid value as we used during creating geom column.
# lib/gpx_phoenix/tracks/import_track.ex
defmodule GpxPhoenix.Tracks.ImportTrack do
alias GpxPhoenix.Tracks.Track
@spec call(gpx_doc: String.t()) :: {:error, %Ecto.Changeset{}} | {:ok, %Track{}}
def call(gpx_doc) do
gpx_doc
|> GpxEx.parse()
|> get_first_track()
|> build_track_geometry()
|> create_track()
end
defp get_first_track({:ok, %GpxEx.Gpx{tracks: [track | _]}}), do: {:ok, track}
defp build_track_geometry({:ok, %GpxEx.Track{segments: segments} = track}) do
multilinez_coordinates = convert_segments_to_mulitlinez(segments)
track_geometry = %Geo.MultiLineStringZ{
coordinates: multilinez_coordinates,
srid: 3857
}
{:ok, track, track_geometry}
end
defp convert_segments_to_mulitlinez(segments) do
Enum.map(segments, fn segment ->
Enum.map(segment.points, fn point ->
{point.lon, point.lat, point.ele}
end)
end)
end
defp create_track({:ok, %GpxEx.Track{name: name}, track_geometry}) do
GpxPhoenix.Tracks.create_track(%{name: name, geom: track_geometry})
end
end
Let's import some example Gpx files. Here are three tracks I recorded during my rides https://github.com/caspg/gpx_phoenix/tree/master/gpx_files. We can import them using Elixir console.
iex -S mix
iex(1)> {:ok, gpx_doc} = File.read("./gpx_files/gdansk-elblag.gpx")
iex(2)> GpxPhoenix.Tracks.ImportTrack.call(gpx_doc)
# same for othe files
Converting track to GeoJSON
GeoJSON format is designed to represent geographical objects and is based on the JSON. It is commonly used in web mapping applications. We can convert our geometry to GeoJSON using PostGIS function.
# lib/gpx_phoenix/tracks/tracks.ex
defmodule GpxPhoenix.Tracks do
import Ecto.Query, warn: false
alias GpxPhoenix.Repo
alias GpxPhoenix.Tracks.Track
# ...other functions
def get_geom_as_geojson!(%{id: id}) do
query =
from(t in Track,
where: t.id == ^id,
select: fragment("ST_AsGeoJSON(?)::json", t.geom)
)
Repo.one!(query)
end
end
Tracks controller
Let's create tracks_controller, tracks_view and corresponding templates. tracks_controller will have two standard CRUD actions and one action, geojson, for fetching track's GeoJSON asynchronously.
defmodule GpxPhoenixWeb.Router do
# omitted code
scope "/", GpxPhoenixWeb do
pipe_through :browser
get "tracks", TracksController, :index
get "tracks/:id", TracksController, :show
get "tracks/:id/geojson", TracksController, :geojson
end
end
# lib/gpx_phoenix_web/controllers/tracks_controller.ex
defmodule GpxPhoenixWeb.TracksController do
use GpxPhoenixWeb, :controller
def index(conn, _params) do
tracks = GpxPhoenix.Tracks.list_tracks()
render(conn, "index.html", tracks: tracks)
end
def show(conn, %{"id" => id} = _params) do
track = GpxPhoenix.Tracks.get_track!(id)
render(conn, "show.html", track: track)
end
def geojson(conn, %{"id" => id} = _params) do
geojson = GpxPhoenix.Tracks.get_geom_as_geojson!(%{id: id})
json(conn, geojson)
end
end
# lib/gpx_phoenix_web/views/tracks_view.ex
defmodule GpxPhoenixWeb.TracksView do
use GpxPhoenixWeb, :view
end
# lib/gpx_phoenix_web/templates/tracks/index.html.eex
<ul>
<%= for track <- @tracks do %>
<li>
<%= link(track.name, to: Routes.tracks_path(@conn, :show, track.id)) %>
</li>
<% end %>
</ul>
# lib/gpx_phoenix_web/templates/tracks/show.html.eex
<h2><%= @track.name %></h2>
Interactive web map
We can use Leaflet.js to render an interactive web map. Before writing any JavaScript code we have to include Leaflet CSS and Leaflet JavaScript files. To make things simpler we can include those files in the show template.
We also need an HTML element that will serve as a container for our map and will hold track-id as a data attribute. We are going to use a track-id to fetch correct GeoJSON.
# lib/gpx_phoenix_web/templates/tracks/show.html.eex
<h2><%= @track.name %></h2>
<div id="track-map" data-track-id="<%= @track.id %>" style="height: 500px; margin-top: 50px;"></div>
<link rel="stylesheet" href="https://unpkg.com/leaflet@1.6.0/dist/leaflet.css" />
<script src="https://unpkg.com/leaflet@1.6.0/dist/leaflet.js"></script>
The only thing that’s left is to create an actual map and render our track on it. Leaflet allows for adding GeoJSON layers. There is also a handy function that will make sure our track fits the map. In this example, I’m using OpenStreetMap as a free tiles provider. In a production app, we should look for some commercial provider.
// assets/js/app.js
function renderMap() {
const trackMap = document.getElementById('track-map')
if (!trackMap) {
return
}
// create leaflet map object
const map = L.map(trackMap)
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
attribution:
'© <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors'
}).addTo(map)
const trackId = trackMap.dataset.trackId
// fetch geojson and add it to the map as new layer
fetch(`/tracks/${trackId}/geojson`)
.then(res => res.json())
.then(geojson => {
const geojsonLayer = L.geoJSON(geojson).addTo(map)
// handy function that makes sure our track will fit the map
map.fitBounds(geojsonLayer.getBounds())
})
}
renderMap()
Finally, we can see our tracks on the interactive map. In case you are wondering, yes, there is Hel in Poland.
Comments
You can reach my via email or discuss on Twitter.
Links
- https://twitter.com/thecaspg/status/1241682445335920645
- GitHub repo containing code used in this blogpost
Top comments (0)