Jochen Topf

A New Website

Sun, 21 Apr 2013 18:14:51 +0200

I have had a personal web page for a long time. I started creating web pages in 1993 when I was a student at Freiburg University. At that time I didn’t have any way to publish it. But when I moved to Karlsruhe later that year I could put up my first real public home page on the university server there. In 1995 my web pages moved to the INKA web server and in 1998 they moved again to my new home in cyberspace on remote.org. In 2005 I added a web page for my “professional self” at www.jtic.de.

As these things are, time for and interest in my own web pages waxed and waned over the years and I often neglected to update them. For many years now my web sites have been totally out of date – I was busy with other things. Two and a half years ago I registered a new domain jochentopf.com and started moving my web pages over. That took longer than I had planned, but I am happy to announce that I finally made it. The new web site is online!

I had decided to join my personal and professional web sites into one. It is often difficult to keep private and commercial projects apart, as my hobbies tend to turn into work and my work influences my hobbies. I have moved over everything from my jtic.de site and it only redirects to the new site now. I moved large parts from remote.org, too, but there is still some stuff to do before I can retire that site. And, of course, there is some new information on the page and more to come.

The new web page got a new logo, a new design and some Javascript gimmicks, we have to go with the times… These days it is actually fun fiddling around with CSS and Javascript. Of course, this blog needed a new design as well, so everything fits together. As it has been the case with my former web sites, most of the pages on my new site are available in English and German.

Upcoming talks at FOSSGIS

Wed, 10 Apr 2013 10:28:23 +0200

The yearly FOSSGIS conference, the German language conference for Open Source GIS, Open Data and OpenStreetMap, will take place from June 12th to June 14th 2013 in Rapperswil, Switzerland near Zurich.

Yesterday the program was announced. I’ll be giving a talk about the OpenStreetMap data model and together with Tim Alder will also talk about our multilingual map project.

There are some interesting topics on the program and I am looking forward to the conference and to seeing you there!

State of the OSM Coastline

Mon, 11 Mar 2013 17:08:48 +0100

A year ago I released the first version of the OSMCoastline program that extracts and assembles coastlines from OSM data. Since then I have learned more about coastlines in OSM than I ever wanted to know, fixed numerous coastline bugs myself and added more features to the software. OSMCoastline has been in production use at Geofabrik and the German map server for more six months now.

OSM Inspector

While assembling the coastline data, OSMCoastline spits out information about errors in the OSM data, such as self-intersections of the coastline or ways that are oriented the wrong way around. In September 2012 I added a coastline view to the OSM Inspector. This view shows those (potential) and real problems. Since I made this data easily available, people have been very diligent at fixing the problems. Unfortunately new problems crop up again and again as people are working on the OSM data who either don’t know how to correctly tag the coastline or they are not paying enough attention to what they are doing.

Currently we have reached a sort of stalemate. Most days the coastline is broken in a few places, but people fix those problems quickly. OSMCoastline automatically fixes some problems (not in OSM, just in its output) and some problems only affect very small areas, so the situation is not too bad. But it needs constant attention. If we don’t keep up with fixes, the coastline data is unusable after a few days.

Lakes tagged as coastlines

When I started work on the coastline there were a lot of small and large lakes incorrectly tagged with natural=coastline instead of natural=water or similar. There were two common cases: Very large lakes (like the Great Lakes in North America) that were created before the introduction of multipolygon relations. Without multipolygon relations there was no way these large lakes could be modelled in OSM. The other case are inlets, lagoons, etc. that were tagged as coastline but they are actually not connected to the sea, maybe there is a dam that cuts them off or so. In some cases it is not immediately obvious whether these features are part of the coastline or separate entities. But by looking closely at the aerial images you can usually see whether there is a connection or not. Sometimes these connections are seasonal, though, or change with the tides, and it is not always clear what the best tagging is.

Since then I added the display of “small lakes” tagged as coastline to the OSM Inspector, they show up as “Questionable” because I want people to use their judgement in fixing them. First I displayed only features with less than 1,000 nodes, those were all converted to proper lakes quickly. Currently I show those kinds of features with less than 10,000 nodes.

There are only a few of those cases left in the world and I am not sure whether it is a good idea to fix them all. Those lakes are huge and having multipolygons that big could make editing the data difficult. Ultimately this can only be fixed in my opinion by adding a proper multipolygon object type to OSM.

You can see the remaining large features tagged as coastlines in the image above and here is a list:

Lake Winnipeg, Lake Winnipegosis, Cedar Lake (Canada)
Lake Nipigon (Canada)
Great Lakes (North America)
Bras d’Or Lake (Canada)
IJsselmeer (Netherlands)
Lake Ladoga and Lake Onega (Russia)
Caspian Sea

There is some debate whether the Caspian Sea is to be classified as lake or as a sea, which might have important consequences for its use by neighboring states, but all the other features are definitly lakes.

Another issue is that properly tagged lakes will not show up in small zoom levels any more, because coastlines are rendered in all zoom levels, but lakes are only rendered in zoom level 6 and up. But this can and should be solved by creating properly generalized water polygons of large lakes that can be used in lower zoom levels.

Antarctica

The coastline of Antarctica in OSM is quite bad and we are working on fixing this. But the quality of the data is not the only issue here. The following map sketch illustrates this issue.

First, the current coastline is cut off at 85.0511° South, so the dark blue area on the map sketch above is lost. The commonly used web maps can’t show anything below this line, so you can’t see this. But if you use a different map projection this is a problem. Second, to support these maps the coastline is “closed” by adding a fake coastline (the “red lines” in the image above). So in OSM the land area of Antarctica actually ends at 85.0511° S, there is a round ocean around the South Pole…

The Antarctica import mentioned above will change the coastline to use the outer line of the ice shelves (the green line in the map sketch, see the wiki for the reasoning). As a nice side effect this will solve the first problem. The second problem is solved by leaving the coastline open (ie not connecting the ends at 180°W and 180°E to anything) and adding a special case to the software using the coastline. OSMCoastline already knows how to handle this case. It internally closes the coastline creating an Antarctica polygon that either goes to the 85.0511° line or to 90° depending on which output projection you have chosen. So the result will be usable in the usual tiled web maps as well as in other maps.

Antimeridian

Similar to the Antarctica case but slightly different is the case at the Antimeridian (180th meridian). The antimeridian crosses Russia, Fiji, and Antarctica. I’ll leave this problem as an exercise for the reader…

Statistics

There are currently about 716,000 ways tagged natural=coastline with together 30 million nodes representing 1.2 million kilometers of coastline. This means that on average there is a node every 41 meters in the coastline, an extraordinary achievement for the OSM community!

There are nearly 480,000 islands, most rather small. The largest single object created from this coastline is the polygon for the land mass of Asia+Africa+Europe containing 4.3 million nodes followed by the 3.5 million node polygon of the American continent.

OSMCoastline

The OSMCoastline tool has been around for a year now and has proven to be very stable. It is easier to install, better documented, runs much faster and has more features than its predecessor coastcheck. Of course as the author of OSMCoastline I am biased but I can only recommend that everybody who still uses coastcheck switch to OSMCoastline.

OSMCoastline does not only generate the coastline as line, but also land and water polygons. Optionally it can split up the huge polygons into manageable pieces. Unlike coastcheck which can only generate land polygons and splits them up in a grid of regularly tiled pieces, OSMCoastline will split the polygons recursively into smaller and smaller pieces until each has less points than a configurable limit.

OSMCoastline counts all problems detected while it runs. It will exit with different return codes depending on whether there were no problems at all, only a few warnings, or too many or larger errors. You can use the return code to decide whether the data is usable at all. This is far better than with coastcheck where you had to manually check whether the data is usable.

Results of a daily run of OSMCoastline are available at OpenStreetMapData.com. In addition to the check done by OSMCoastline itself, as explained above, OpenStreetMapData.com also checks a rendered bitmap of the coastline data against last days rendering to find out if there are large changes. If there are too many changes, OpenStreetMapData.com will keep the old data. It is better to keep older data than giving out a possibly broken coastline.

The German map/tile server has been updated automatically each night from OpenStreetMapData.com for several months now without any reported problems.

Generalized Coastlines

Christoph Hormann has been working on generalizing coastlines. These look very nice and I think we should work towards using them on OSM maps. We probably need generalized lakes, too, and as coastlines often coincide with boundaries, generalized boundaries, to make this really look perfect.

Antarctica Import

Wed, 06 Mar 2013 20:16:48 +0100

I have written before about the mapping of Antarctia in OSM. Since then there has been a lot of activity. Christoph Hormann has prepared better Antarctica data for import and has documented the planned import extensively on the wiki. Peter Körner has created a web map in Polar Stereographic Projection.

Some of that work was done during the recent Karlsruhe Hack Weekend where we also discussed the tagging for the new Antarctica features.

At the hack weekend, Christoph and I were interviewed on this topic by Peter Körner and Andreas Hubel for Radio OSM, the German language OSM podcast, you can listen to the interview here (Interview is in German).

There are still things to discuss and do before we can do the actual import, but I am very optimistic that we will soon have better Antarctica data.

Using Taginfo to Create Map Feature Tables

Mon, 25 Feb 2013 18:31:41 +0100

The Map Features wiki page is one of the oldest pages on the OSM wiki and it has been getting larger and larger over time. And more and more wiki templates are used to create it and its language variants. Very similar tables can also be found on many “Key:*” pages on the wiki to list the tags used with this key.

The Problem

All these lists are hard to maintain. You have to dig through multiple levels of wiki templates to understand what is going on. It is practically impossible to keep this information consistent throughout the different places in the wiki.

At the recent Karlsruhe Hack Weekend Andreas (Andi) Hubel and I had a good discussion about this issue and possible ways to improve the situation.

Andi did some statistics and found out that there are about 3800 templates and about 15000 cases where one template includes another template! Much of that is due to the Map Features pages. This is not only hard do understand for a human, it is hard on the MediaWiki software, too, because changes in one page affect other pages and MediaWiki has to keep track of all this, re-render them when needed etc.

Using Taginfo

An obvious solution would be to auto-generate the tables on the Map Features pages using taginfo. Taginfo gets most of the necessary information already from the “Key:*” and “Tag:*” wiki pages. (The only information missing is the little map snippets that show how features are rendered. This picture is currently not referenced from the infoboxes on the key and tag pages where taginfo gets its information.) If taginfo generates those tables, we’d make sure the information is always consistent. And we can get rid of lots of unwieldy templates. What we do need in this case are wiki pages for all tags with info boxes on them so that taginfo can find all this information.

So I gave it a try and added a new API call to taginfo that allows easy access to this information in JSON format and I created a test web page that takes this information and displays it in a form similar to the current Map Features pages. You can give it a try at mf.jochentopf.com.

You can set the language you want the descriptions in by entering the usual language code. If the description is not available in that language it falls back to the English description.

To choose which tags to show there are several possible ways. If you just enter a key such as highway or boundary you will get all tags with that key that are documented in the wiki. Or you can enter a comma-separated list of tags in the form KEY=VALUE. If you just give a VALUE in the list it uses the KEY from the tag before. So boundary=administrative,national_park,landuse=residential will show you both boundary tags and the landuse=residential tag.

All of this is just a quick proof-of-concepts. There are multiple problems still to be resolved. Feel free to play around and give me feedback.

What now?

To completely replace the existing Map Features lists we have to decide where to place those autogenerated lists. We could either include them in the wiki somehow or they could live on a separate web page (for instance the taginfo web page).

If we integrate them in the wiki users will see the lists they are already used to and they can edit the text around those lists which would be a big plus. But it needs more work to make this possible. We’d probably need some kind of MediaWiki plugin that creates those tables from the JSON delivered by taginfo. We could include an iframe in the wiki page that gets its content from an external site. We already do this for those little taginfo info boxes on some pages in the wiki. Unfortunately we can’t make this seemless because the main page doesn’t know how big the iframe should be, so this isn’t that great a solution.

Maybe you have an idea how to do this?

Antarctica in OpenStreetMap

Mon, 04 Feb 2013 12:33:08 +0100

OpenStreetMap wants to map the whole world, but there is a whole continent that has been neglected a bit: Antarctica. Of course there isn’t much there besides rocks, ice, and penguins and not too many people live there. But still, it is a huge area and it should appear properly on our maps.

I have looked into the current situation and what could be done and want to write that up here as a first step towards improving the situation.

Mercator Projection

The Mercator projection generally used in online maps only covers the area between about 85.0511 degrees South and 85.0511 degrees North. While this is not a problem in most cases, it obviously is a problem for Antarctica. JOSM can edit data South of -85.0511 degrees when you change the projection from Mercator to “WGS84 Geographic” (EPSG:4326), but Potlatch can’t. And the usual aerial images aren’t available, because they also use the same Mercator projection.

The Coastline

If the source tags are right, the coastline seems to have been mostly imported from GADM. Apart of the licensing issues, this data seems to be old and not very accurate. The real coastline dips below 85.0511 degrees South in the Ross sea, but in OSM it is cut off there, probably because not many people are actually creating maps not using the Mercator projection. There are newer and better sources for the Antarctica coastline around and it should not be too difficult to throw the current one out and replace it by a better one.

Ice Shelves

There are huge ocean areas around the land mass of the continent that are covered in ice shelves. They should be tagged and appear on the map because they are large features and they affect the way we perceive Antarctica.

There is no tag for ice shelves yet, there is natural=glacier, but that is not the same. I propose adding a tag natural=ice_shelf for this. There are sources for the borders of the ice-shelf, so it wouldn’t be a big problem to add them.

Available Aerial Images

Bing and Mapbox seem to be using older and not very detailed Landsat images. There are newer Landsat images with better resolution available, see below. Landsat images don’t go below 82.5 degrees South because of the orbit of the Landsat satellites. The images from MapQuest Open are better, they are from Landsat and mostly from NASA’s BlueMarble NG.

Data Sources

If you want to have a look around, maybe try out some of the data yourself, here are some pointers.

I have asked Frederik and he has added Antarctica to the list of OSM extracts that are generated daily and published on download.geofabrik.de. It is not much data and you can load all of it into JOSM, though it will be a bit slow then.

The current OSM coastline is available from OpenStreetMapData.com, it is usually updated daily.

There are several sources of reasonably current data that seem to be quite accurate and detailed.

The Mosaic of Antarctica (MOA) contains good satellite images for Antarctica and, derived from it, shapefiles. You can look at them and download the shape files. It seems that Natural Earth Data derives their Antarctica coastlines from these. The shape files can’t be used directly for import, though, because the vectorization is bad and the data contains lots of ziggzaggy lines.

Another data source could be the Landsat Image Mosaic Of Antarctica (LIMA) and derived data. You can look at the images or the Research Atlas. There are also downloads and WMS available.

(Thanks to Christoph Hormann for these links and insights into the data.)

There are also several more links on the wiki on the Antarctica page. I haven’t looked at all of them yet.

Licensing

Of course we have to look into the details, but licensing should not be a big issue. Almost all of this data is ultimately derived from NASA satellite images or other satellite data which is in the Public Domain. Either NASA themselves or other US government or research agencies have been processing this data to create the data that is available at the places mentioned above.

Displaying data

I can display all of this data in EPSG:4326 in Quantum GIS (QGIS). The Antarctic Polar Stereographic Projection EPSG:3031 is much better, though. Unfortunately there are some rendering problems with it in QGIS. I haven’t tried Mapnik or other renderers yet. This is something I must explore some more.

Bogus data

There is one other problem with OSM data in Antarctica: Just like the spot in the Atlantic with 0 degrees latitude and 0 degrees longitude, Antarctica seems to attract some bogus data. Most probably from software bugs, but it is also a likely spot to add bogus things for the fun of it without anybody noticing. Last time I looked, I found (and removed), among other strange things, a restaurant, a restroom, and a pub called “Very Cold” somewhere in Antarctica and a parking lot and a tree from failed imports at the South Pole.

Where to go from here?

I think we can probably just throw away the existing coastline and import the coastline and ice-shelf data from NaturalEarth. We don’t actually need that much detail in Antarctica. If and when we need it we can always add the more detailed data from MOA either in the whole of Antarctica or just in selected areas. This is much less work than cleaning up the better MOA data and making it ready for OSM and we’d still have a large improvement over the current situation. If it turns out that there are areas where the data currently existing in OSM is better than the NaturalEarth data, we should, of course, keep that, or check with the better MOA data.

So far these are just some notes from my research. There is more work to be done. I hope this article will start a discussion. Please tell me if you have any other knowledge, ideas, insights. Before actually doing any kind of import, I will, of course, write up in some more detail what is planned and discuss that according to the Import Guidelines.

Public Transit Mapping

Wed, 23 Jan 2013 16:09:51 +0100

A map and routing service for the Kathmandu public transport system. And how the data was collected. This is why I love OSM.

Taginfo News

Thu, 10 Jan 2013 12:09:52 +0100

It has been nearly a year since I did some serious work on taginfo. Taginfo works well, but there has been and still is a backlog of todo items. And I get new ideas on how to improve it all the time. So I spent the first days of the new year doing some work on it. A lot of this work has been “behind the scenes”, improving code here or there, so there isn’t actually that much to see. (If you are taking a look remember that you might have to clear your browser cache for all the goodies to show up properly.) But here are the larger improvements:

Switch from Protovis to D3

Taginfo had been using the Protovis Javascript library for the graphs you can see on some pages. But Protovis is not developed any further. Instead the author started from scratch and created D3 which is even more awesome!

I switched all the graphs over to the new library. This took some work without much visual change, but should make it easier to build on in the future.

I also introduced a new and improved shiny colorful tag cloud on the front page based on D3 and the supercool D3 Wordcloud code. See the image above.

Support for Images from Wiki

The pages on the OSM wiki describing keys and tags often have an image on them depicting what this key or tag is about. I now properly get the meta data about these images from the wiki and show the images in the user interface. Where images are available they are shown on the key and tag pages in the Overview tab and if you mouse over the image links in the Wiki tab they are also shown.

I added support for the images because of a question from Tom MacWright on osm-dev. He wants to use those images in the new iD Javascript editor. This is now easy to do as the image links etc. are also exported via the taginfo API. Which brings me to the next chapter…

New API Version

Taginfo always had an API that allows everybody to get at all the juicy data in taginfo. I internally use this API to show most of the information you see on the site. But the API was organized badly and never fully documented. So now I have re-organized the API a bit, given it the new version 4 and documented every call. The changes for users aren’t really that great, so it should be easy to change your code. In most cases you will only need to change the URL. The API documentation optionally shows deprecated API calls, too, and tells you which new call you should use instead of the old calls.

The old API versions will be around for a while so you have some time to switch. But they will disappear at some point, so if you are using the API, consider switching sooner than later.

Caching of Wiki Pages

Taginfo reads thousands of wiki pages to get its information. This takes a lot of time and it strains the wiki server. The new version now caches the wiki pages and only reloads them if they have changed.

Status of the Multilingual Maps Project

Wed, 19 Dec 2012 15:09:07 +0100

It is high time I’ll write something about the status of the Wikipedia Multilingual Maps project I have been working on for many months now.

On November, 30 2012 I announced a demo of the multilingual map at mlm.jochentopf.com. This demo allows the user to choose the language or languages they want to see the map labels in. More then 500 people looked at the map in the first 24 hours and I got a lot of very positive feedback. Based on this feedback I improved the demo a bit, for instance allowing several labels at once.

As planned all of this is implemented based on the Mapquest Render Stack. Unfortunately my changes are not merged into the upstream tree yet, but you can get everything in my fork of the software. For the demo/test this software is running on a virtual machine on bessel.openstreetmap.de, the tile server used for the German OSM map. Eventually the software should run on a Wikipedia server and the multilingual maps should be available for everybodies use. There are currently some issues to be resolved inside Wikipedia to get those servers.

The Wikipedia project itself is finished now, all its goals were implemented. I have written up a final report that describes the work we did in more detail. But of course that doesn’t mean that everything is done. There is always more to work on.

I am currently working on some changes to the Mapnik renderer that would allow flexible data transformation plugins to be added. These plugins can (among many other things) be used to do transliteration on-the-fly. My tests with that show promise and I hope to roll out transliteration to the demo server soon.

I am also working on on the demo/test-server, doing changes here or there, whats mainly missing is expiry of tiles, but that should stabilize pretty soon and then we are planning to open it up to a beta-test where others can use those tiles, too.

An Area Datatype for OSM

Mon, 26 Nov 2012 17:04:42 +0100

One of the biggest problems with the OSM data model is that there are no real polygons. Polygons or multipolygons are “simulated” using closed ways or multipolygon relations and several other methods. These multipolygon relations are often broken. Some of the problems resulting from this non-existence of multipolygons are affecting many people, for instance with broken coastlines and country or other boundaries.

This is not a new problem. In March 2011 I created the wiki page The Future of Areas as a focal point for documentation and discussions around this and others built on it. There are some interesting ideas there, but until now no clear solution could be found. Every proposed solution has many drawbacks. At the Karlsruhe hack weekend in June 2012 a group of us discussed this problem during dinner and we made some good progress. Unfortunately nobody wrote down afterwards what we had talked about. In this blog post I try to build on this discussion and try to formulate the proposal.

Problems

Before we go into a solution we have to make clear what the problems are with current (multi-)polygons. First there are several ways of handling what is essentially a single issue. You can have simple (smallish) polygons using closed ways. For larger polygons (like whole countries) or multipolygon you need multipolygon (or boundary) relations. And there are some cases where we don’t use either of those but have more special solutions again, coastlines ( which essentially describe land (or water) polygons) and river areas modelled by “riverbank” ways come to mind. Having several solutions for essentially the same problem is wasteful and complicated.

The second problem is that some polygons are really small (say a rectangular building) and some are huge, like a whole continent. Any solution must take this into account. The third problem are the many broken multipolygons. There have been tools around for years to help fix those broken multipolygons, but they get broken again and again. We (the OSM community) has failed to produce a solution that actually works.

For any solution to be viable we must break with some conventions in the way our data model and our API work. First, the way we edit OSM data has always been: Download complete objects, edit them, and upload the new version of the objects. There have been discussions about partial downloads and partial edits, but that has never materialized. But to be able to edit huge objects like country-sized multipolygons, it will be essential that we are able to download and edit parts of an object.

Another time-honored tradition has been that the central OSM database and the API are rather dumb. They do only minimal checking on the data that is being uploaded, allowing broken geometries to be uploaded, for instance, ways with a single node or polygons with self-intersections. If we ever want to have a chance of having country-sized objects without geometry errors, we have to change this. The central database (or the API) has to make sure that we don’t create objects with those problems.

Area objects

So, when we look at those requirements it becomes actually relatively straightforward to think about a solution for the polygon problem. Let’s define an “area” object very similar to a “way” object. A list of references to nodes and some tags. In addition we require that the first and last references nodes must be identical.

How should the API look for editing such things? A request goes to the API to download a given bounding box that the user wants to edit. To be able to work with an area we need all nodes inside that bounding box plus at least the next node after the last nodes in the bounding box. If we have the complete area all is fine, if not we need one more thing: We need to know which side is “inside” and which is “outside”. To define this we just define that the nodes in an area must be sorted so that they always go around the polygon in a clockwise order. (This is arbitrary, it could be counter-clockwise, but it is a common convention to use clockwise). The editor can now always draw the area properly (inside the bounding box it has downloaded), together with the tags it has all information it needs.

And this representation has another interesting property: If the whole polygon is valid (no self-intersections of the boundary etc.) and if all edits are restricted to the bounding box and the result of those edits is valid, the whole polygon will still be valid. If the user uploads some changes we don’t have to check the validity of the whole polygon, only of the changes. This is very important because otherwise the check would be too expensive.

Why is the whole polygon still valid? I am quite sure that this is the case, but I don’t have a rigorous mathematical proof. Somebody should probably find one to make sure we are not forgetting anything here.

There are, of course, many questions left. For instance I have described the “area” datatype just for simple polygons and I have only described outer rings. I think the area datatype should allow multiple outer and inner rings, so it should describe complete multipolygons as defined by the simple feature definition.

We might also need some API functions to migrate data, says change a closed way into an area. This might be done client-side but just adding an area and removing the way, but we’ll see which approach is better.

Migration

Migration of existing OSM data to the new model is relatively easy, because there is nothing taken away from the current data or API. First we need to implement the new area object in the central database and all the needed API calls. We then need to change editors to allow working with the new areas including download and upload through the new API calls. All other OSM software such as osm2pgsql is changed to allow the new-style areas in addition to old-style areas.

Once all of this is in-place and people can actually work with the new area datatype we automatically convert old-style way-based polygons and multipolygon relations to the new style. We can start with simple polygons (such as all closed ways that are valid polygons and are tagged with building=yes and nothing else), and work our way to more and more complex cases. What’s left over are polygons that are in some way broken or other corner-cases that have to be fixed manually. Mappers can just load those cases into an editor, add a new polygon and remove the old ways/relations. Once most of the data is migrated we remove support for old-style polygons from osm2pgsql and other such software. Data that’s not yet converted will not show up on the map any more which should give the boost to the community to finish converting those cases, too.

Where do we go from here?

We obviously need a lot more discussion in the community to make sure this is a valid proposal and that we can solve all the problems with it that need solving. Maybe I have overlooked something important and it doesn’t work at all? And once we are reasonably sure we have a valid way to go we need buy-in from all relevant parties.

I think it would probably make a good diploma thesis (or something like it) to work out all the details of this new data model, the API and the changes needed to the different kinds of software working with OSM data. (Feel free to approach me if you are interested.)

Update: Several people have pointed out that this could break if there are edges of the polygon going through the downloaded bounding box but without nodes inside the bounding box. Of course we have to download the nodes at the end of those edges, too. Then what I am describing above still works. And of course we have to figure out how to implement all of this efficiently.