[JT] Jochen Topf's Blog

osmium

Expedicious and Exact Extracts with Osmium

There is too much OSM data! Of course we always want more, more, more data. But when all you need is the data for your city or country, handling 34 GByte of (densly compressed) OSM data for the whole planet can be overwhelming. So one of the most often needed tools for working with OSM data is something that will cut out the region you are interested in from the huge planet file.

| Read more…

Tags: openstreetmap · osmium


Osmium news

Its high time I write something about the news in the Osmium world. After 10 days or so of multiple releases in short succession things have quieted down some and I think we have something more stable now.

| Read more…

Tags: openstreetmap · osmium


Libosmium and Osmium tool releases

Yesterday I released version 2.9.0 of libosmium and version 1.4.0 of the Osmium command line tool. There are quite a few significant changes.

| Read more…

Tags: openstreetmap · osmium


Node Locations on Ways

OpenStreetMap has a somewhat peculiar data model. OSM ways don’t contain the locations of the points they go through, but only references to nodes, which in turn contain their location. This model has some advantages, but it also has a major disadvantage: For almost all uses of OSM, like drawing maps or calculating routes, the software has first to find the locations of all nodes in a way, before it can do anything with it. This is cumbersome and potentially expensive. To do this quickly, the lookup needs to be done in main memory without any disk access, but to store the locations of all nodes of a full planet file we already need about 32GB RAM. But maybe we can create a shortcut, so that we can do this once, instead of everybody doing this again and again?

| Read more…

Tags: openstreetmap · osmium


Renumbering OpenStreetMap

The number of objects in OpenStreetMap is growing and with it the number of unique IDs. For node IDs we have gone over 4 billion now. Soon they will not fit into an unsigned 32 bit integer any more (232=4294967296).

| Read more…

Tags: openstreetmap · osmium


Related amenity tags in OSM

Matias Dahl has looked at historic OSM data and done some analysis on changing use of the amenity tag including some pretty cool graphs. Nice work and potentially very useful. And he used node-osmium for it. Now, how do we get something like this into a continuously updated service, maybe even into taginfo?

Tags: openstreetmap · osmium · taginfo


Osmium is growing up

It has been a long time since I wrote something about Osmium in this blog. Considering that I have worked on Osmium and related software almost every day for many months now, this is a huge omission. I want to make up for this a little bit with this posting.

| Read more…

Tags: openstreetmap · osmium


Osmium Command Line Completion

When I started using the ZSH shell years ago, it was because of its superior command line completion features. Since then, the Bash shell has improved a lot, and probably has the same features, but I am used to ZSH now. And when a command doesn’t have completion it feels cumbersome to use. So, of course, I wanted to have completion for the osmium tool.

| Read more…

Tags: osmium · zsh


Osmium Talk at SOTM-EU

Last weekend we had the SOTM-EU here in Karlsruhe. I had a lot of fun, but didn’t get enough sleep, something you’ll notice in the video from my Osmium talk. Oh well, not everybody can have their talk on the first day.

Tags: conference · event · openstreetmap · osmium · sotm-eu · talk


Osmium News

Over the last months I have been busy working on a project for Mapbox. As part of that project I have spent a lot of time improving Osmium. And not just a few changes here or there but a more or less complete redesign based on the experience of developing and using Osmium for the last nearly three years now.

| Read more…

Tags: c++ · dev · mapbox · openstreetmap · osmium


Osmium MultiPolygon Refactoring

Osmium has had support for assembling multipolygons from relations for a long time. But the code had been lifted from an earlier Geofabrik project and was rather messy, hard to extend, and it contained several memory leaks. In the last month or so I finally chewed through the whole thing bit by bit and made it prettier.

| Read more…

Tags: c++ · dev · openstreetmap · osmdata · osmium


Osmium Refactoring

I have done some long-overdue refactoring of the Osmium code which simplifies Osmium itself and simplifies its use. And should make it faster to compile, because only those parts that are really needed are included.

| Read more…

Tags: c++ · dev · openstreetmap · osmium


Osmium Packages

I just noticed that someone has created official Debian packages for Osmium (libosmium-dev, osmjs). They have also found their way into the new Ubuntu 12.04 version. Unfortunately they are already outdated. It is fine if you only want to use osmjs, but if you do any development based on Osmium, I suggest you use the version from github.

| Read more…

Tags: dev · openstreetmap · osmium


More Coastlines Stuff

For a while now I have been working on OSMCoastline adding features bit by bit. (See this blog article for some background info.) OSMCoastline now closes small gaps in the coastline and it can split up the huge polygons into smaller ones. You can create land polygons or water polygons depending on your needs and you can have it all in WGS84 or in the usual Mercator projection used for tiles web maps (EPSG 3857). So it does a lot more than the coastcheck application used for these things before. And the software is still a lot faster.

| Read more…

Tags: openstreetmap · osmcoastline · osmium


Magazine article on osmjs/tilemill

Today the April 2012 issue of the German computer magazine “iX” was in my mailbox. It contains my four and a half page article on how to create your own maps with OSM data. I describe how to use Osmium/osmjs software to extract the data and Tilemill to style and create the maps.

Tags: openstreetmap · osmium


FOSSGIS talks

I gave two talks at the FOSSGIS 2012 conference last week. The first, OpenStreetMap in Zahlen und Karten, was not recorded and I will not release the slides as they will be difficult to understand and easy to mis-understand without my words. I will probably re-use the content for a few blog articles here in the future.

| Read more…

Tags: conference · event · fossgis · openstreetmap · osmium · talk


FOSSGIS 2012

Next week is the FOSSGIS conference in Dessau. I’ll be speaking there about OpenStreetMap in Zahlen und Karten and Das Osmium-Framework. Together with Tim Alder I am also organising a community session to discuss the Wikipedia Multilingual Maps Project. See you in Dessau!

Tags: conference · event · fossgis · multilingual maps · openstreetmap · osmium · rendering · talk


OSM Coastlines

Every map of the world has to take into account that there are land areas and water areas. You could have huge polygons for each continent and paint them on a blue background. But handling those huge polygons is difficult. OpenStreetMap solves this problem by having ways tagged with “natural=coastline” and the convention that the land is always on the left side of this way. So a continent or an island is surrounded by one or more ways in counter-clockwise order. It is much easier to edit those coastlines instead of large multipolygon relations or something like it. Coastlines are the only case where OSM has this special rule.

| Read more…

Tags: openstreetmap · osmcoastline · osmdata · osmium


Hacking Weekend II

The Karlsruhe Hacking Weekend is over. 18 people were there at least part of the time. I had a lot of fun and even got a little bit of work done: As mention I worked on Taginfo and on the second day I worked a bit on the statistics for my upcoming talk at the FOSSGIS conference. I am using Osmium and full-history planet dumps to see what has been happening in OSM. I’ll publish some code and graphics at some point.

| Read more…

Tags: dev · event · openstreetmap · osmium


Osmium ToDo list

The Osmium framework has been around for about a year and a half now and has grown to include a lot of functionality. But there is still much to to. In this blog post, I have collected some of those things. There is no particular order to the following list.

| Read more…

Tags: dev · javascript · openstreetmap · osmium


OGR support for Osmium

Osmium now includes OGR support. OGR is a popular Open Source library for converting different GIS storage formats into each other. It supports PostGIS, Shapefiles, Spatialite, and many other formats. I had started working on OGR integration many weeks ago, but didn’t quite finish it. I have now added the last bit: Support for OGR Multipolygons. This opens up a lot of new possibilities for Osmium users.

| Read more…

Tags: dev · gis · ogr · openstreetmap · osmium


Unit Testing in C++

Osmium is getting bigger and bigger and doing more and more. But testing the code is still done manually and ad-hoc. Not good. I have used some testing frameworks in other languages, but never in C++. As usual, Boost comes to the rescue. I recently added some test code using the Boost Unit Test Framework which (at least for now) does all I need.

| Read more…

Tags: c++ · dev · openstreetmap · osmium · testing


State of the Map EU

The organisers of the State of the Map Europe have published the programme and it looks very interesting. I will give a talk about Taginfo and a talk about Osmium, the two projects I currently spend my free time on.

| Read more…

Tags: conference · event · openstreetmap · osmium · taginfo · talk