I am a heavy git user. I am actively developing in about half a dozen repositories at the same time and have many more I occasionally check out or add to. I have files lying around that need to be added, changes to be committed, commits to be push, all in several branches, local and remote. And then there is the stuff I stashed months ago and forgot…

I have been writing lots of software for the OpenStreetMap project and there is always one problem: How to test it? There is a huge variety of data in OSM, every conceivable way of tagging, every way of getting the data slightly (or not so slightly) wrong, will sooner or later appear. We need to test file formats, OSM objects syntax and semantic, changing data, cases with failing referential integrity or invalid geometries and much much more. But just coming up with all those test cases is a huge task and implementing those tests is even more work.

Over the last months I have been busy working on a project for Mapbox. As part of that project I have spent a lot of time improving Osmium. And not just a few changes here or there but a more or less complete redesign based on the experience of developing and using Osmium for the last nearly three years now.

Osmium has had support for assembling multipolygons from relations for a long time. But the code had been lifted from an earlier Geofabrik project and was rather messy, hard to extend, and it contained several memory leaks. In the last month or so I finally chewed through the whole thing bit by bit and made it prettier.

I have done some long-overdue refactoring of the Osmium code which simplifies Osmium itself and simplifies its use. And should make it faster to compile, because only those parts that are really needed are included.

I just noticed that someone has created official Debian packages for Osmium (libosmium-dev, osmjs). They have also found their way into the new Ubuntu 12.04 version. Unfortunately they are already outdated. It is fine if you only want to use osmjs, but if you do any development based on Osmium, I suggest you use the version from github.

The Karlsruhe Hacking Weekend is over. 18 people were there at least part of the time. I had a lot of fun and even got a little bit of work done: As mention I worked on Taginfo and on the second day I worked a bit on the statistics for my upcoming talk at the FOSSGIS conference. I am using Osmium and full-history planet dumps to see what has been happening in OSM. I’ll publish some code and graphics at some point.

The Karlsruhe Hacking Weekend is in full swing. About 15 people are sitting around, munching apples, pizza and Gummi Bears and talking about things “somebody” should do “someday”. And we do some actual work, too. I fixed a few bugs in Taginfo, Sven and others are working on updating the German map style. Some people are talking about obscure Mapnik config options. No idea what the other people are doing.

I recently started writing my own RSS and Atom feed reader. I’ll write another blog post about why and how I am doing that. Today I want to focus just on one part of the job: Reading the RSS and Atom feeds.

The Osmium framework has been around for about a year and a half now and has grown to include a lot of functionality. But there is still much to to. In this blog post, I have collected some of those things. There is no particular order to the following list.

Time for the next OpenStreetMap hack weekend. Hope to see you there.

Osmium now includes OGR support. OGR is a popular Open Source library for converting different GIS storage formats into each other. It supports PostGIS, Shapefiles, Spatialite, and many other formats. I had started working on OGR integration many weeks ago, but didn’t quite finish it. I have now added the last bit: Support for OGR Multipolygons. This opens up a lot of new possibilities for Osmium users.

Osmium is getting bigger and bigger and doing more and more. But testing the code is still done manually and ad-hoc. Not good. I have used some testing frameworks in other languages, but never in C++. As usual, Boost comes to the rescue. I recently added some test code using the Boost Unit Test Framework which (at least for now) does all I need.

Martijn van Exel is organizing a hack weekend on June 10-12 2011 at the Linuxhotel. I am planning to be there. After many hack weekends in London this ist the first such event in Essen after a long time. The first OSM event at the Linuxhotel was the Essen Developers Workshop I organized there four years ago. Has it already been that long?.

It is quite amazing what can be done with good data and current map rendering technology. Mapnik and other renderers have continually pushed the envelope and there are many examples of beautiful maps out there.

The Unicode character set contains somewhat over one million code points from 0 to hex 10ffff. That wasn’t always so. Unicode started out with only 16 bit characters, or about 65000 code points. At some point it was decided that that wasn’t enough and the version 2.0 released in 1996 switched to the larger character set.

I have wanted to play around with NodeJS for a while. I had some time on the weekend so I implemented a small tileserver for Tirex using NodeJS.

Some month ago I introduced Taginfo and mentioned there in passing that the statistics collection is done based on a C++ framework called Osmium. I promised another post and I am delivering on this promise now. I have been working on Osmium on and off over the last 6 months or so and, while it is far from perfect, I think it could be actually useful for other people now. So lets dive right in. What is Osmium and what is it good for?

The OpenStreetMap project gets bigger every day. More people edit the data which leads to more data in the database and this to more people using the data. And with the growing database it gets harder and harder to see whats in there. Gone are the days when the common mapper would need to know maybe a dozen different tag keys, each with just a handful of typical values. Today people tag parking spaces and public transport, power lines and penguin pools in their local zoo. Some people are valiantly trying to keep up by documenting tags in the wiki, but sometimes that seems to be a lost cause. There are only about 550 wiki pages describing a key, but there are about 20,000 different keys in the database (about one third only used once, many will by typos) not to mention the 50 Million different tags. What are all these tags? How are they used and by whom? The endless discussions on the mailing lists about what tags to use are partly because the world is complex and fitting everything in our tagging model is not easy, but they are also in part a symptom of this confusing situation.