Generalization project wrapup

For the last six months I have been working on the project to add generalization support to osm2pgsql. This project was funded by the German Ministry of Education and Research (and organized by the Prototype Fund). Many thanks to them for giving me this opportunity!

(This post is part of a series about generalization of OSM data.)

There was a lot of work in the last weeks, getting everything done and documented, preparing a presentation for the official demo day and writing the final report. I also had to prepare my talk for the upcoming FOSSGIS conference. So it took a bit longer to write this blog post but I want to give the mini series of posts I have written in the last months a sort of conclusion.

Generalization is a hard problem and we are a long way from a satisfactory solution. But I think we have done a good step here. I have implemented several generalization strategies, see the other blog posts in this series for the details on those. The original plan was to add all this to osm2pgsql itself, but we have decided to put them into a separate program (called osm2pgsql-gen) for the time being. The new program “lives” in the osm2pgsql repository and is compiled alongside osm2pgsql (if you have all dependencies installed). The reason to split this off is simply that the code is rather new and the interface is still somewhat experimental. It is still the plan to fully integrate this with osm2pgsql once we get some operational experience with it.

The documentation is already fully integrated with the rest of the osm2pgsql documentation. It is still a bit rough around the edges though. I have also added a directory for example config files, but so far there is only a single example there.

Here is a map of Europe that uses some of the techniques developed in this project. You can see the generalized landuse areas, the urban areas and the labelled cities chosen using the discrete isolation algorithm:

I am currently doing some longer-term tests with updates of OSM data. Getting the minutely updates from OSM and processing them continuously. The forest layers (one original and generalized layers on three different zoom levels) update in a few seconds on our test machine, so it is looking good. I have set up a tile server [update: not available any more] which is directly fed from the database, no caching involved to get a sense how fast that setup is. (And no, there isn’t a bug in the program. The blocky look of Canada and some other parts of the world is due to the spotty data there.)

While the project is now officially concluded I will keep working on this, time permitting. I hope to get into lots of conversions about this with users of osm2pgsql so that we can see in which direction this can evolve. And I have many ideas of other generalization algorithms I want to try.