Preventing the Flooding of the Planet

I have written before about my work on the software doing the assembly, repair, and conversion of the OSM coastline data. I wrote the Osmcoastline software and I run it daily to create shapefiles with land and water polygons available for download on OpenStreetMapData.com [Update: Discontinued June 2019, now available from osmdata.openstreetmap.de]. For a while this data has now been used on the main OSM map. Unlike with the old process, this new process usually makes sure the map is updated within a few days when the coastline changes.

But the coastline is still broken in one or more places nearly every day. It gets fixed quickly (thanks to the coastline view at the OSM Inspector) but somebody always manages to break it somewhere else soon. Osmcoastline can fix some problems, like small gaps in the coastline, but it can’t fix them all.

To make the coastline data reliable to use in spite of those problems, some scripts check the generated coastline each day and compare it with last days coastline. For this check a map containing only the land polygons is rendered into a 2048×1024 bitmap and a bit-by-bit comparison is done with the bitmap of the last day. If the differences are too large, there must be some problem and the files I have for download at openstreetmapdata.com are not updated. Only when the checks are successfull will the download files be updated. Users of the files can use the If-Modified-Since-Header to check for this and only download the checked data.

At the last hack weekend in Karlsruhe I created a movie that shows how our map would look like from day to day, if I didn’t do that check. You see islands and whole continents disappearing and reappearing, often only because there was a small error somewhere in the coastline.

The movie shows each day from October 2012 to the end of September 2013. Look out also for the Antarctica import in March 2013.

You can also download the movie in full 2048×1024 resolution. Thanks to Christoph Hormann for helping with the creation of the movie.