Introduction 👋

So last year, I created a web app called kreuzungen.world that calculates the number of waterways crossed by a gpx route. It was a fun little project to build and led to some interesting encounters.

The app was built using javascript in a way that required no backend.. Thats right, all the data fetching and processing was done in the browser. (No server costs 💸).

It uses the Overpass API to fetch the waterways data from OpenStreetMap and then uses turf.js to calculate the intersections between the route and the waterways. Pretty simple, right?

What’s been surprisingly rewarding is seeing people actually use it! From Japan to Argentina to New Zealand to the USA (where one kayaker uses it to track river crossings). It’s humbling to see something I made for fun, being used by people across the globe.

kreuzungen.world isn’t changing the world, but it is a great example of how open source software and open data can be used to create something (crossing rivers is not going to save lives, but my shred-buddies find it interesting, so that’s something).

The powerful client 💪

I want to point out my genuine surprise at how fast the browser does the geospatial processing… It just works, even on old devices, the performance is good enough. We are talking a couple of seconds for most routes. Including fetching the waterway data via OSM, render the calculated intersecting waterways on a vector map. Waterway fans are happy, I am happy too 😁

But, I am not completely satisfied… Recently I have had a craving for speed in my life, don’t know why, but I feel it. And fittingly I decided to revisit this problem and see how fast I could push this thing 🚴

I had an Idea 💡

River data doesn’t change that often, so by pre-processing the data and using a storage format optimized for querying it should be much faster My train of thought was something like this 🤔

• Pre-download waterways instead of waiting for the Overpass API 💾
• Use duckdb alone for fast geospatial queries 🌍🦆
• Implement R-Tree indexing to minimize intersection search space 🎯
• Apply two-step filtering: bounding box, then precise intersection 🔍
• Lightweight API Layer with endpoint for GPX upload 💻

Well folks, buckle up, because I took that idea and ran with it and it turned out to be a sprint! 🏃

The Setup 🛠

The entire solution is available on GitHub. Feel free to adapt it for your own needs.

The solution has two parts:

• Preparing the database
• Wrapping it in an API

Actually thats a lie, there is another part…

• The benchmarking! ⏳

I also included a benchmark with some different gpx files, because, well timing things makes sense (when you’re obsessed with speed 🎽).

The entire solution is built using open source tools and libraries and is available on GitHub. Feel free to explore and adapt it for your own projects! If you have any suggestions or feedback, please reach out. I’m always eager to learn and improve.

The project structure looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

├── README.md
├── prepare_waterways_data.py   -- data pipeline to prepare the database
├── server.py                   -- fastapi wrapper
├── benchmark.py                -- benchmark script
├── benchmark_logs              -- benchmark logs
├── data                        -- data directory
│   ├── filtered/               -- filtered waterways (.pbf)
│   ├── parquet/                -- filtered Waterways (.parquet)    
│   ├── pond.duckdb             -- local duckdb database
│   └── raw/                    -- raw data osm (.pbf)
├── quelle                      -- dbt project
│   ├── dbt_project.yml
│   ├── models                  -- transformation logic
│   └── profiles.yml            -- duckdb setup
└── test_data
    └── gpx                     -- benchmarking gpx files

The “Big Data” Preprocessing Pipeline 🚰

Let me walk you through the pipeline, which is at the heart of the solution, that transforms raw OpenStreetMap data into the DuckDB table optimized for spatial querying. This is the source of the Wasserwege API 🌊

1. Download OSM Data: Using extracts from Geofabrik rather than processing the entire planet file.

2. Filter Waterway Features: Using Osmosis, filter for waterway features from the OSM data. This dramatically reduces the data size.

3. Convert to GeoParquet: Using ohsome-planet. The format is columnar and much more efficient for analytical queries. This Java tool is amazing, it does a great job at converting the OSM data into a format that is easy to work with.

4. Build database for querying: I leverage dbt with the DuckDB adapter. This framework brings structure to data transformation workflows. It nicely separates the data transformation logic from the data engineering configuration. Makes a setup that’s easy to maintain and extend and minimizes boilerplate code.

R-Tree Indexing 🌳

The most critical optimization is the creation of an R-Tree spatial index to organize the geometries in a hierarchical tree structure.

When a query like “find all waterways that intersect with this route” is executed, the R-Tree allows the database to quickly eliminate vast portions of the dataset without checking each waterway individually. This reduces the time complexity from O(n) to something closer to O(log n).

An index can be created with a simple SQL statement, thanks to the duckdb SPATIAL extension:

1

CREATE INDEX waterways_geom_idx ON waterways USING RTREE (geom);

This single line of code provides dramatic performance improvements for spatial queries.

Imagine trying to find a group of friends at a festival… good luck if you have to search through the entire crowd. But if you know they will be at the beach stage, you can skip out the masses and search through only the people in the near vicinity. The R-Tree index is an ordering of data such that you quickly narrow down the search space to just the relevant geometries.

DBT-ing it all together 🧩

The waterways.sql model combines the transformations and creation of the R-Tree index.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

{{
    config(
        materialized="table",
        file_format="parquet",
        location_root="../data/processed/",
        pre_hook="DROP INDEX IF EXISTS waterways_geom_idx;",
        post_hook="CREATE INDEX waterways_geom_idx ON {{ this }} USING RTREE (geom);",
    )
}}

with waterway_features as (
    select
        osm_id as id,
        st_geomfromwkb(geometry) as geom,
        tags['name'] as waterway_name,
        tags['waterway'] as waterway_type
    from {{ source("osm", "waterways") }}
    where tags['name'] is not null
)

select *
from waterway_features

The output of this all is a local duckdb file pond.duckdb with a table waterways that contains all the waterways, configured ready for fast querying.

The API 🎮

The API is built using FastAPI, a modern web framework for building APIs with Python. My aim was to keep this lightweight and hand off the heavy lifting to the database. I added two endpoints to the API:

• /healthcheck: A simple endpoint to check if the API is running and healthy and reports the number of waterways in the database.
• /process_gpx: This endpoint accepts a GPX file, parses it, and finds the waterways that intersect with the route. It returns the results in a JSON format.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

@app.post("/process_gpx")
async def process_gpx(file: UploadFile = File(...)):
    """Process GPX file and find waterway intersections"""
    overall_start_time = time.time()

    # Read and parse the uploaded GPX file
    contents = await file.read()
    gpx = gpxpy.parse(contents.decode())
    
    # Convert to linestring and find intersections
    linestring = gpx_to_linestring(gpx)
    crossings = find_waterway_crossings(linestring)

    # Return results with timing information
    return {
        "processing_times_ms": {...},
        "total_crossings": len(crossings),
        "crossings": crossings,
    }

The Query ✍️

The core query that powers the intersection detection is remarkably simple (thanks to the R-Tree index and DuckDB’s spatial functions):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

WITH route_geom_cte AS (
    SELECT ST_GeomFromText($1) as geom,
           ST_Envelope(ST_GeomFromText($1)) as bbox
)
SELECT
    w.id,
    w.waterway_name,
    w.waterway_type,
    ST_AsGeoJSON(ST_Intersection(w.geom, r.geom)) as intersection_geojson
FROM waterways w, route_geom_cte r
WHERE ST_Intersects(w.geom, r.bbox) -- First fast filter with bounding box
  AND ST_Intersects(w.geom, r.geom) -- Then precise intersection

This query uses the created index to quickly identify potential intersections, then it performs the exact spatial operation only on those candidates.

Quantifying the Gains 📈

To measure the performance gains, I created a benchmarking script (benchmark.py) that tests different GPX files against the API. The results showed a drastic improvement:

You can see the performance of the API in action. The benchmark script runs 10 different GPX files, each with varying lengths and complexities, and measures the time taken to process each file, and it runs before the old browser-based solution even finishes a single route.

This represents a biggggg in processing speed compared to the browser-based solution!

Conclusion: Performance is a Journey, Not a Destination 🧘

This experiment shows that with different tools and techniques, we can dramatically improve the performance of applications. What was already “pretty fast” in the browser is now much faster because of a change in architecture 🦆

I’ll be honest, I don’t even have a feeling of what a “fast” optimized response to this problem should be 😲

There are so many ways this problem can be solved using a computer, and each approach has its own characteristics.

I didn’t explore other optimizations, like using a different database or alternative indexing strategies. I could have written low-level code in Rust, but that would take far more time than I had available for this project.

We are lucky to have such fast tools available to us today to use for free, and can build on top of really well made software. This saves time ❤️

Takeaways: Performance is a Journey, Not a Destination 🧘

It’s deeply satisfying to see abstract ideas transform into real performance gains. What started as a theoretical hunch about spatial indexing and data processing became a real and measurable improvement. It’s a humble reminder that studying these concepts can actually lead to practical benefits when applied to the right problem.

Not everything in life needs to be fast, but in situations where it matters, a well-chosen index and thoughtful data pipeline might just be the solution you need! 🏁

Standing on the Shoulders of (Open) Giants 🙇

None of this would be possible without the incredible open-source tools and libraries that are available today.

OpenStreetMap Contributors: Every river and stream was manually mapped by volunteers in one of humanity’s most impressive collaborative projects 💚

Geofabrik: Provides free daily OSM extracts, saving countless hours of redundant processing. Thank you! 🙇

Osmosis: When OSM data is like IKEA (an overwhelming maze with too many options when all you need is cup), this tool helps filter just what you need 🕵️‍♀️

ohsome-planet: Transforms OSM’s simply-elegant-but-awkward data model (just nodes with tags and pointers) into the GOAT GeoParquet format 📦

DuckDB: Turns your laptop into a high-performance geospatial data warehouse 🦆

dbt, gpxpy, FastAPI: The building blocks that tied everything together into a coherent system 🧰

Intro 🪐

I found myself at the State of the Map Europe event! An annual gathering of OpenStreetMap enthusiasts. I am relatively new to the OSM world and saw it as a fantastic chance to get involved and learn more about the community and the projects happening in the space. Oh my, what a big space it turns out to be! OSM is everywhere.

Some special talks, for no specific reasons:

• AI-assisted mapping by the Humanitarian OSM Team (HOT🔥)
• Flowing Connections: Mapping rivers & streams with WaterwayMap.org
• Supporting Mobility Transitions: OpenStreetMap’s Contribution to Public Administration Data Needs
• Minutely vector tiles for the community

It was a great event! Kudos to the organizing team! I met a ton of cool people, soaked up some serious knowledge, and all while enjoying the vibrant city of Łódź.

A lot of folks I met were curious about my journey and had many questions about how I managed to transport myself to the event.

This blog post spills the beans on how I prepared to ride my bike from Berlin to SOTM, held in Łódź, Poland. 500km¹ in 2 days 🚴‍♀️🌕🚴‍♂️.

The Bike Setup ⚙️

I’m a firm believer that you can enjoy a bike tour on pretty much any bike with the right mindset.

For this journey, I aimed to go fast and efficiently, stay comfy, and not be totally done by the time I arrived in Łódź. The way would be mostly flat, so weight wasn’t a concern. I would mostly ride on the road to save time.

I chose to break the ride into two days, with an overnight in a cheap hotel in Poznan. This meant I could leave the tent, sleeping setup, and cooking gear behind. It would just be me, my bike, and the essentials.

Here are the key areas I focused on to optimize my setup for the journey:

• Comfort
• Aerodynamics
• Rolling resistance

Comfort 😌

Meet Trustini, my trusty long-distance steed.

I have done many long rides on this bike and know it fits me well. A gravel bike with a relaxed geometry, built around a buttery smooth titanium frame, and a lot of history. We have a good relationship built on years of trust and going places together.

For long distances comfort, the contact points are where the magic happens: saddle, handlebars, and pedals. Discomfort tends to accumulate here, potentially causing trouble if not addressed. Everyone’s body is different, and it is best to experiment with different setups and find out what works for you personally.

I use a narrow Brooks C13 Cambium saddle with a bit of flex and team it up with my most comfortable bib shorts from Santini that include a butt-saving “C3 seat pad with anti-shock gel inserts”. This combination has proven itself on many long rides.

For the handlebars, I use the thickest bar tape I can find. More cushion absorbs the vibrations that come through the bars better. I also wear padded gloves.

My friend Michael gave me some aero bars to try out. This gives me more hand positions to switch between, helping avoid any nerve damage² from the long days in the saddle.

I use clip-in pedals with stiff shoes that fit me well. This means I can pedal more efficiently and have the force evenly distributed around my whole foot.

The combination of these things hopefully means I can ride for hours without any discomfort 🤞

Aerodynamics 💨

When cycling above 15km/h, the biggest thing slowing you down is air resistance or drag.

Considering I am much bigger than the bike, the biggest source of drag is me. Changing your position on the bike can make a big difference.

While I will try to stay as aero as possible, a comfortable position is key. So I opted for a tight-fitting jersey. Not because it looks cool, but because it’s practicool.The reason the pros wear them is that it’s faster to have skin-fitting clothes than things flapping around.

Wearing a helmet actually reduces drag and makes you faster, as proven by Greg LeMond in the 1989 Tour de France³. People actually compare helmets to see which one is fastest⁴.

I wear an Oakley Ar05 Race helmet which does a good job. I also use sun glasses with a big lens to protect my eyes from the sun, wind, and bugs. The frame of the glasses matches up with the helmet and reduces drag even further. Marginal gains, but hey, I will take every saving I can!

The rest of my stuff was stashed in my bike bags. Top tube bag + frame bag + so-called “Arse Rocket” saddle bag. The bags are positioned to avoid increasing the frontal area and avoid any extra turbulence. According to some sources, the bags might even help me stay aero⁵.

It was my first time using aero bars, and wow, it was a game changer! My intention was to give myself different positions to rest my hands. But oh boy, when I was in the aero bar position, I felt like I was piercing through the air. It was also super comfortable to be able to switch between different positions. I will definitely be using them again on long rides.

Plus, I could mount my phone on the aero bars in a chiller position, optimal for navigation and changing up tunes on the go.

Rolling Resistance 🛞

Another crucial contact point is the tires. Tires are the only part of the bike touching the ground and are the biggest source of rolling resistance.

Traditionally, the fastest (aka the least rolling resistance) tire is slick and narrow, inflated to high pressure.

This idea is evolving with new perspectives, lower pressures, and the industry producing tubeless tires and wider rims. According to new data, a wider tire at a lower pressure can be faster than a narrow tire at a high pressure, especially on rougher surfaces.

I like Continental GP5000 tires, and according to the rolling resistance data⁶, the 32mm version, which I am using, is not much worse than the skinny 25mm version, and is one of the faster tires available.

Using 32mm, fatter and softer tires also means more comfort, especially on the rough stuff, which is important for long rides.

With all this setup, I should be rolling with ease. The only thing left to do was to pack my bags and hit the road 🚴

Packing List 🧳

The Route 🗺️

I used Komoot to plan the route. I put in my start and end locations and set Poznan as a waypoint 📍 to break up the journey. I tweaked the route by adding a waypoint in a national park to avoid the main road, and that was it.

Komoot provides the surface type and elevation expected along the route to give you a good idea of what to expect and helping you pace yourself.

Komoot Premium includes a feature that shows the expected weather along the route at the time you are expected to be there. It also displays the sunrise and sunset times, which are super useful for deciding what to take with you on a tour.

I saved the route offline on my phone, strapped it to my handlebars, and was ready 🛫

The Wind 🌬️

Nature plays a huge factor in cycling, but it’s beyond your control. Sometimes you get unlucky and have a rainy headwind all day; sometimes you get blown off your bike by a gust of wind, but that is all part of the fun.

I used windy.com to check the wind forecast, and it looked like the wind gods would be blowing in my favor for once!

Get in! I would have a tailwind all the way to Łódź.

It was forecasted to be dry and hot, which means a lower air density and even less drag. I got lucky.

The Journey 🚴‍♂️

I left Berlin around 10:00 AM. I had a laptop with me, which I dropped off at Dominik’s place on the outskirts of Berlin (big thanks for taking care of it!). From then on, I was heading east!

The first 100km were super smooth. I was flying! The sun was shining, and I was moving through the forests of Brandenburg with a tailwind all the way to the border. I was cruising on the aero bars at around 30km/h without much effort.

I went over the Oder/Odra River, which marked the international border between Germany and Poland. I do love a good Schengen border crossing. You just glide through without being stopped or checked. I probably wouldn’t have even noticed if I didn’t know I was crossing a border. Especially as someone who used to be an EU citizen, I truly appreciate the freedom of movement.

The road quality changed as soon as I crossed the border. The roads became rougher and bumpier. Such is life. It would be interesting to analyze the road quality data from OSM to see if there is a difference between countries. I imagine that the surface_type completeness in Polish street data in OSM is lower than in Germany, but that is a topic for another time.

I went along a Park narodowy (aka a national park) and saw some beautiful birds flying around, many storks. I also saw a few deer and a big frog.

One remarkable thing I noticed was how clean Poland was. There was no trash anywhere. I didn’t see a single piece of litter on the road, and the fields and forests were well-kept.

A lake provided the perfect spot to cool off and eat some food. Poland is beautiful in the summer.

The afternoon heat was intense, and I was cooking, my skin slick with a slimy mix of salty sweat and sunscreen lotion. But whatever, I was moving fast.

I spent the night in Poznan, a beautiful city with a lot of young people and a lot of history. I arrived, had a shower, and headed out to explore the city and have something to eat. The city was alive with people and music. I almost got swindled in a strip club, but that is a story for another time.

After a big breakfast, I hit the road again. The second day was again super smooth. I had a tailwind all the way to Łódź. I was flying!

A sudden pothole caused a snakebite⁷. No biggie, I had a spare tube and fixed it in no time and was back on the road cruising again.

With about 50km left, I encountered a group of Polish gravel riders. They were, let’s just say, a lively bunch. They invited me to join their ride. We raced into Łódź together, they showed me a few sights, and we finished the journey sharing stories and beers. A perfect ending to an epic ride.

The Food 🍲

A few people asked me how I would eat on the road.

A key to long-distance cycling is fueling your body with enough calories to sustain the effort you’re putting out. If you don’t, you might find yourself in an unfortunate situation called “bonking”⁸.

I like to take things like Bananas, muesli bars, croissants with me and keep many stashed in my top tube bag. I eat these regularly while riding to maintain my energy levels. You can’t eat too much on a long ride, so keep snacking!

In Poland, you find “Sklep Polski” everywhere, which are small local shops selling everything you need and can resupply. The owners were always nice and refilled my water bottles. I also wanted to taste some traditional Polish food, so I stopped at a couple of restaurants along the way and enjoyed some delicious pierogi.

Since it was so hot, I needed to drink a lot of water on the ride, like 6 litres a day. However it was never a problem to find clean water en route. I took some isotonic tablets with me to stay hydrated.

Takeaways for Long-Distance Cycling 📝

• Bike Fit: Ensure your bike fits you well. Small adjustments can make a huge difference in comfort and performance.
• Fueling: Eat, eat, eat! Keep your energy levels up by eating regularly. Stock up regularly on the road and always have some snacks on hand. Stay hydrated!
• Gear: Invest in gear that works for you and prioritizes comfort and efficiency. Tight fitting clothing and good tires can significantly enhance your ride.
• Route Planning: Use tools like Komoot to plan your route. Knowing the terrain and expected weather conditions helps you prepare better.

Conclusion 🏁

State of the Map 2024 was fantastic! I had a great time and learned a lot, which could fill another blog post.

The journey itself was a highlight and went incredibly well.

Traveling across Poland by bike gave me an opportunity to see the country evolve. I saw the small villages, the fields, the forests, the rivers, and the big cities. I had a lot of time to see it and take it all in.

Cycling has many uncountable benefits, but for me, it’s the best way to travel and straight-up fun. I recommend for anyone interested to get out there and give it a go!

The Problem

I use a lot of different maps applications and often end up “map hopping”. That involves finding a place on one mapping app and then trying to locate the same place on another app for “reasons”.

Sometimes this is a smooth and quick process, but other times it can be a real pain fumbling around to digitally triangulate¹ and locate the same place on both maps.

You might encounter two very different looking basemaps, which often leads to squinting and searching for a uniquely-shaped river bend on both maps, using that to line things up and then inevitably finding another bend in a river and doubting if you’re even looking at the same thing.

The Solution

MapSwap, is a simple yet ingenious tool I recently discovered through the OSM US Slack community. It provides a way of switching between mapping application whilst preserving the viewport. This means no more time spent manually aligning things and is a game changer 🔥.

Encoding the viewport in the URL

Many mapping services use a common pattern of encoding the viewport settings (latitude, longitude and zoom level) into the URL using the hash property.

Consider for the sake of some examples, the undisputed arbitrarily-chosen center of the World², The Royal Observatory in Greenwich, London with co-ordinates 51.4769° N, 0.0000° E and a arbitrary zoom level of 13.

Here’s how different mapping services encode this information into the URL:

• OpenStreetMap: https://www.openstreetmap.org/#map=13/51.4769/0.0000
• Komoot: https://www.komoot.com/plan/@51.4769000,0.0000000,12.000z
• Windy: https://www.windy.com/?51.477,0.000,13
• Google Maps: https://www.google.com/maps/@51.4769,0.0000,13z

Most mapping libraries have built-in support for this feature out of the box.

1
2
3
4
5
6

let map = new Map({
  container: 'map',
  center: [0, 51.4769],
  zoom: 13,
  hash: true // SET THIS TO TRUE!
})

How MapSwap works

MapSwap is a little utility that you can add to your web browser, a Bookmarklet³. When you click the “MapSwap” bookmark, the browser executes a bit of javascript that parses the {x} and {y} coordinates and {z} zoom level from a map via the URL, using the window.location⁴ object. It then opens another tab, allowing you to select from various mapping applications. The selected application will open configured with the same coordinates and zoom level, essentially preserving the original viewport.

The file map.js houses a JSON array of mapping application entries.

Each entry contains the name of the application, a URL template string, an icon URL, and an array of tags.

A entry for Komoot would look something like this:

1
2
3
4
5
6
7

{
  name: "Komoot",
  template:
    "https://www.komoot.com/plan/@{y},{x},{z}",
  icon: "https://www.komoot.com/favicon.ico",
  tags: ["commercial", "outdoor", "osm", "bike", "routing"],
}

The template string uses the placeholders {x}, {y} and {z} to insert the coordinates and zoom level into the URL. The tags are used to filter the applications in the UI.

The full list of parameters is documented in the MapSwap README.

• {x} - Longitude
• {y} - Latitude
• {z} - Mapbox & MapLibre style Floating-point Zoom level
• {Z} - Google style Floating-point Zoom level (one greater than Mapbox/MapLibre style zoom)
• {Zr} - Leaflet style integer Zoom level (one greater than Mapbox/Maplibre style zoom)
• {zr} - Integer Zoom level (don’t know if any maps use this)

The Mapping community

The online mapping community is incredibly welcoming and friendly. I am repeatedly impressed by the quality of the tools and the willingness of developers to help each other out.

I noticed that Komoot wasn’t supported in MapSwap, so I messaged the creator Daniel Schep and put in a PR to add Komoot to the tool. It was a pleasure to chat with Daniel, and the PR was merged in no time.

Conclusion

There are many different mapping applications out there, each having pros and cons. Combining them together is incredibly useful, and MapSwap makes this easy. It means less fumbling between different maps trying to pinpoint the same location and that’s a win in my book.

The next level to this would be syncing different maps viewports in real-time.

Imagine planning a big winter ski-touring-bike-packing trip using Komoot on the left side of the screen and then both snow conditions and ski touring routes on the right side, and everything moving in sync.

Sounds awesome, however browser security restrictions make this a bit tricky… a topic for another post.

Introduction

When using a map on a digital device, it is important that it is fast to load and the map interactions are seamlessly smooth. A fast loading map will go by unnoticed as “everything works” leading to a natural feeling experience. A slow loading map will get in the way of a good user experience, leading to frustration, grief and a bad taste in the mouth.

Vector maps are being used more and more on the web. They offer a faster and more interactive experience and more ways of being styled.

I am using maplibre-gl, which is a vector map library in a hobby project and interested in finding out which map style is the fastest to load.

Map drawing 101

A vector map library needs a recipe to draw a map on the screen. The recipe will have instructions telling the mapping library where to request the vector data from and how to render that data to the screen. Rendering instructions include what parts of the data to draw, what colors to use and what order to draw the layers of data in.

When you pan and zoom to a specific part of a vector map, the library will know based on this recipe, where to request the body of data that should fill up the screen and how to style it.

A good designer aka a cartograpaher, will design a style in a way that it shows off the best parts of the data with certain balance of space and color, producing something that looks appealing and allows easy reading of the data.

What makes things fast or slow?

The time taken to display a map on the screen will depend on where the data comes from and how complex the rendering instructions are.

It makes sense that a style that is simple and requests a little data from somewhere close to the user will load faster than a style that is complex and requests a lot of data from somewhere far away from the user.

A Peek inside a Style document

Here is a taste of a style à la MapTiler Basic Light**

(note: I have omitted most parts to simplify the example)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

{
  "version": 8,
  "id": "basic",
  "name": "Basic Light**",
  "sources": {
    "openmaptiles": {
      "url": "https://api.maptiler.com/tiles/v3/tiles.json?key={YOUR_API_KEY}",
      "type": "vector"
    },
    "maptiler_attribution": {
      "attribution": "<a href=\"https://www.maptiler.com/copyright/\" target=\"_blank\">&copy; MapTiler</a> <a href=\"https://www.openstreetmap.org/copyright\" target=\"_blank\">&copy; OpenStreetMap contributors</a>",
      "type": "vector"
    }
  },
  "layers": [
    {
      "id": "background",
      "type": "background",
      "paint": { "background-color": "rgba(224, 224, 208, 1)" }
    },
    {
      "id": "landcover_grass",
      "type": "fill",
      "source": "openmaptiles",
      "source-layer": "landcover",
      "paint": { "fill-color": "rgba(192, 213, 169, 1)", "fill-opacity": 0.4 },
      "filter": ["==", "class", "grass"]
    },
    {
      "id": "landcover_wood",
      "type": "fill",
      "source": "openmaptiles",
      "source-layer": "landcover",
      "paint": { "fill-color": "hsl(82, 46%, 72%)", "fill-opacity": 0.8 },
      "filter": ["==", "class", "wood"]
    },
    {
      "id": "water",
      "type": "fill",
      "source": "openmaptiles",
      "source-layer": "water",
      "layout": { "visibility": "visible" },
      "paint": { "fill-color": "hsl(205, 56%, 73%)" },
      "filter": ["all", ["!=", "intermittent", 1], ["!=", "brunnel", "tunnel"]]
    },
    {
      "id": "building",
      "type": "fill",
      "source": "openmaptiles",
      "source-layer": "building",
      "paint": {
        "fill-color": "rgba(212, 204, 176, 1)",
        "fill-opacity": 0.6,
        "fill-antialias": true
      }
    },
  ],
  "glyphs": "https://api.maptiler.com/fonts/{fontstack}/{range}.pbf?key={YOUR_API_KEY}",
  "bearing": 0,
  "pitch": 0,
  "center": [0, 0],
  "zoom": 1
}

It a very simple style that only shows a few of the features you would expect of a map. It has a background color, grass and wood areas, water, buildings and some text.

There are two sources of data, one for the map data and one for the attribution. This informs the library where to request data from the tile server with address https://api.maptiler.com/tiles/v3/tiles.json?key={YOUR_API_KEY}.

The layers key is where the rendering instructions are. Each layer has an id, a type, a source (in our case it is always the single openmaptiles source) and some paint instructions.

The background gets loaded, polygons of grass gets drawn with a opaque green, water gets drawn in blue, but not when it goes through a tunnel, building footprints rise up.

Paint instructions can get quite complex, and contain many conditional rules and transformations. Things that can be controlled are the size lines, opacity of fills and the font of the text.

Where does the actual vector data get served from?

See the sources key in the above style document. This is where the vector data is served from. The data is served in a format called vector tiles. These are small chunks of data that are ready to be drawn on the screen. The data is normally served in a format that is easy to draw and easy to style.

The ideal place to get the data is the clients device storage. This is fast as the data is ready to be used straight away without requesting and downloading it over a internet connection. But this is not practical for most use cases.

The next best thing is a server vector tile server. This server can be hosted by a tile service provider, or it can be hosted by yourself. There are lots of tools out there to roll your own solution. The Awesome vector tiles repo on github is a good place to start.

Most people will use a third party provider for ease and other reasons instead of DIYing. These providers will have servers all over the world that will serve the data to the user close to where they are.

Where can you get one of these style recipe from?

There are many different places to get vector map styles. Providers offer them. Some of them are free, some of them are paid. Some of them are open source, some of them are closed source. Some of them result in a fast loading experience, some do not.

Different providers offer different styles and it is not always clear which one is the fastest, the Usain Bolt of map styles.

It is also possible to write a custom style, and there are many tools out there that can help you. Maplibre-gl has a style spec that you can use to write your own style. MapTiler has a online style editor that you can use to create your own style.

The Experiment

I used playwright, a browser automation library to launch a chromium browser, initialize a map with a style and time how long it took for the relevant loaded event to fire. This was done for a number of different styles.

The experiment was run on my local machine (A 10 year old HP Pavilion) with a 50mbps internet connection somewhere in Germany.

All the code is available in this repo if you want to replicate the experiment.

Note: Your mileage may vary depending on your hardware and internet connection.

Styles tested

The Open Street Map Wiki lists different vector tile providers. I tested the following:

• MapTiler
• Stadia Maps

Code used to profile the styles

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113

import asyncio
import os
from playwright.async_api import async_playwright

MAPTILER_API_KEY = os.environ.get("MAPTILER_API_KEY")
STADIA_API_KEY = os.environ.get("STADIA_API_KEY")

if MAPTILER_API_KEY is None or STADIA_API_KEY is None:
    raise ValueError(
        "MAPTILER_API_KEY and STADIA_API_KEY environment variables must be set"
    )

STYLES = {
    "MapTiler - Backdrop": f"https://api.maptiler.com/maps/backdrop/style.json?key={MAPTILER_API_KEY}",
    "MapTiler - Basic": f"https://api.maptiler.com/maps/basic/style.json?key={MAPTILER_API_KEY}",
    "Maptiler - Bright": f"https://api.maptiler.com/maps/bright/style.json?key={MAPTILER_API_KEY}",
    "Maptiler - Dataviz": f"https://api.maptiler.com/maps/dataviz/style.json?key={MAPTILER_API_KEY}",
    "Maptiler - Landscape": f"https://api.maptiler.com/maps/landscape/style.json?key={MAPTILER_API_KEY}",
    "Maptiler - Ocean": f"https://api.maptiler.com/maps/ocean/style.json?key={MAPTILER_API_KEY}",
    "Maptiler - OpenStreetMap": f"https://api.maptiler.com/maps/openstreetmap/style.json?key={MAPTILER_API_KEY}",
    "Maptiler - Outdoor": f"https://api.maptiler.com/maps/outdoor/style.json?key={MAPTILER_API_KEY}",
    "Maptiler - Satellite": f"https://api.maptiler.com/maps/satellite/style.json?key={MAPTILER_API_KEY}",
    "Maptiler - Streets": f"https://api.maptiler.com/maps/streets/style.json?key={MAPTILER_API_KEY}",
    "Maptiler - Toner": f"https://api.maptiler.com/maps/toner/style.json?key={MAPTILER_API_KEY}",
    "Maptiler - Topo": f"https://api.maptiler.com/maps/topo/style.json?key={MAPTILER_API_KEY}",
    "Maptiler - Winter": f"https://api.maptiler.com/maps/winter/style.json?key={MAPTILER_API_KEY}",
    "StadiaMaps - Alidade Smooth": f"https://tiles.stadiamaps.com/styles/alidade_smooth.json?api_key={STADIA_API_KEY}",
    "StadiaMaps - Alidade Smooth Dark": f"https://tiles.stadiamaps.com/styles/alidade_smooth_dark.json?api_key={STADIA_API_KEY}",
    "StadiaMaps - Alidade Satellite": f"https://tiles.stadiamaps.com/styles/alidade_satellite.json?api_key={STADIA_API_KEY}",
    "StadiaMaps - Stadia Outdoors": f"https://tiles.stadiamaps.com/styles/outdoors.json?api_key={STADIA_API_KEY}",
    "StadiaMaps - Stamen Toner": f"https://tiles.stadiamaps.com/styles/stamen_toner.json?api_key={STADIA_API_KEY}",
    "StadiaMaps - Stamen Terrain": f"https://tiles.stadiamaps.com/styles/stamen_terrain.json?api_key={STADIA_API_KEY}",
    "StadiaMaps - Stamen Watercolor": f"https://tiles.stadiamaps.com/styles/stamen_watercolor.json?api_key={STADIA_API_KEY}",
    "StadiaMaps - OSM Bright": f"https://tiles.stadiamaps.com/styles/osm_bright.json?api_key={STADIA_API_KEY}",
}

async def time_style(style_name, style_url):
    html_content = f"""
    <!DOCTYPE html>
    <html>
    <head>
        <title>Vector Map Style Profiler</title>
        <script src="https://unpkg.com/maplibre-gl@3.6.2/dist/maplibre-gl.js"></script>
        <link href="https://unpkg.com/maplibre-gl@3.6.2/dist/maplibre-gl.css" rel="stylesheet">
    </head>

    <style>
        html {{
            height: 100%;
        }}

        body {{
            height: 100%;
            align-items: stretch;
            margin: 0;
            padding: 0;
        }}

        #map {{
            flex-grow: 1;
            min-height: 100%;
            max-height: 100%;
        }}
    </style>

    <body>
        <div id="map" style="width: 100%; height: 100%;"></div>
        <script>
            const startTime = performance.now();
            const map = new maplibregl.Map({{
                container: "map",
                style: "{style_url}",
                center: [0, 51.4769], // Greenwich meridian
                zoom: 10,
                maxZoom: 18,
                minZoom: 5,
            }});

            map.on('load', (e) => {{
                const endLoadTime = performance.now();
                loadTime = endLoadTime - startTime;
                window.loadTime = loadTime;
            }});

        </script>
    </body>

    </html>
    """

    async with async_playwright() as p:
        browser_type = p.chromium
        browser = await browser_type.launch()
        page = await browser.new_page()

        try:
            await page.set_content(html_content)
            await page.wait_for_function("window.loadTime", timeout=30000)
            load_time = await page.evaluate("() => { return window.loadTime; }")
            print(f"{style_name}: {load_time}")

        except asyncio.TimeoutError:
            print(f"Timeout occurred for {style_name}")

        except Exception as e:
            print(f"An error occurred for {style_name}: {str(e)}")

        finally:
            await browser.close()


for k, v in STYLES.items():
    asyncio.run(time_style(k, v))

All code is available in the mapStyleProfile github repository.

The Results

Conclusion

Now there are some numbers to quantify the different map styles speed.

Remember speed isn’t everything, and a good map experience is a combination of many things. A fast loading map is just one part of the puzzle, along with space and color, compromises may have to be made to get the best overall experience.

Don’t just take my word for it, test styles out yourself online now

I made a online tool that lets you paste a style url into it and it will time how long it takes to load the map.

All source code is available in the same github repository.

Future ideas

This was a simple experiment to get some numbers on the different map styles, only looking at two providers.

There are a few things that could be done to improve the experiment:

• Add more styles from different providers to get a better idea of the landscape.
• Profile the different parts of the map loading process to break down ingload time
• Add more “real world” interactions to the experiment and see how the styles perform under different conditions.
• Set up a github action to run the experiment on pull requests and provided a central place to see the results.