Libosmium and Osmium tool releases

Yesterday I released version 2.9.0 of libosmium and version 1.4.0 of the Osmium command line tool. There are quite a few significant changes.

The biggest improvement is that OPL files can now not only be written, but also read. OPL files contain a simple ASCII-only representation of OSM data with one object per line. They look like this (but in one line):

w106 v3 dV c8681961 t2011-07-10T10:26:31Z i3112 uHDietze
 Tname=Meraner%20%Straße,oneway=yes,highway=residential
 Nn442757,n1356245842,n1356245836

The format is designed to be easy to create and easy to parse with typical UNIX command line tools or scripting languages.

Read support now means completely new use cases. You can, for instance, easily change tags. Lets say you have a software that doesn’t know about the highway=living_street tag, so you want to change it into the similar highway=residential:

osmium cat data.osm.pbf -f opl | \
  sed -e 's/highway=living_street/highway=residential/' | \
  osmium cat -F opl -o output.osm.pbf

When you try this out, you’ll notice another new feature. Osmium now shows progress bars in many cases. They are actually a feature of libosmium, so it is easy to add them to your libosmium-based programs.

If you want to check that the command did the right thing, you can use the new diff command:

osmium diff data.osm.pbf output.osm.pbf -f debug,color | less

It will show you the differences between the two input files:

This is only a simple example, but Libosmium and the Osmium command line tools offer great building blocks that can be used to solve much more complex problems by using the different pieces together in different combinations.

And while working on features I have also managed to improve processing speed here and there. Libosmium was always fast, but lately it has become even faster. The current version can update a planet file in PBF format with a days worth of changes in 17 minutes (benchmarked on a Xeon CPU E3-1275 @ 3.60GHz with 4 cores and 64GB RAM). Compare this to Osmosis, which needs over an hour, and osmconvert, which needs about 50 minutes.

There are a lot more additions in the Osmium command line tool. The new commands derive-changes creates change files with the changes between two OSM data files. With merge you can merge any number of OSM data files into one. That is very useful when you downloaded some extracts from Geofabrik and want to merge them into one big file.

I have also worked on the documentation for all this. In additional to the manual pages for all the Osmium tool commands there is now a more introductory manual giving an overview of the things that are possible. It is not complete, but an easier entry into the world of the Osmium tool than the manual pages.

On the Libosmium side I have added more example programs and added detailed comments to the source files that help with understanding what’s going on. There is still a lot to be done on the documentation front, but together with the concepts manual, the library manual, and the reference docs there is now a lot of information for developers who want to use Libosmium.

Libosmium and the Osmium command line tool are available for download now, start at osmcode.org. Debian packages for Unstable are already available for the new versions. They will show up in Testing shorty, and backports for Jessie will also be available.

Have fun trying out Osmium and tell me about what you are doing with it.