Introducing Osmoscope

There are many tools that help with quality assurance in OpenStreetMap and they come in many flavours. One popular class of these tools are web sites showing the OSM map in the background and (usually switchable) overlays with markers for different types of problematic OSM data. They usually allow you to jump directly into the OSM editors to fix the problems. The most prominent of these tools are Osmose, the OpenStreetMap Inspector, and KeepRight. But there are many more of these, often written for very specific use cases. All of these tools have the drawback that they are “closed systems”: The person developing the tool and/or running the web service decides which layers there are. The creation of those layers is often tied more or less closely to the tool showing the layers. At the recent Karlsruhe hack weekend I released a new tool, Osmoscope, of that same class, but with much more openness built in.

When you go to osmoscope.jochentopf.com you can try out the tool. It is currently more in a proof-of-concept stage, somewhat usable but buggy. Just looking at the site you’ll notice a lot of similarity with the OSM Inspector. That’s no accident. I wrote the OSM Inspector many years ago (but I am not involved in it any more) and I still like many of the design decisions I made back then. But OSM technology and web technology has evolved. This has allowed some changes, most notably the overlay layers are not bitmap layers any more. Instead GeoJSON or Vector Tile layers are used for overlays.

But the biggest change is not immediately obvious. The largest problem with the OSM Inspector was always the tight integration between client and server. Both are programmed and run by the same person. Every time you wanted to change something or add a new layer, you’d have to do changes on the server. With Osmoscope the server side and the client side are totally separated. The server can be anything that serves web pages. There isn’t any “intelligence” needed on the server side, a dumb web server serving plain pages is enough. This lowers the barrier of entry for new data providers who can concentrate on preparing the overlay data and don’t have to think about the display of the data or other client issues. On the other hand it means that people who want to improve the client can easily install it on their own system, change it and test it with any existing data out there.

Any user of the web site can decide which overlays they want to see. There are a few overlays available when the page is opened, but you can add more by entering the configuration. There you can type in a URL that points to a description of a new layer. Osmoscope will then add this layer to its interface. You can try this with [not available any more]. At the moment the new layer will be lost if you reload the page, but in the future I am planning to add a feature that the browser will remember the layers you added. So over time you can built up a list of the layers you are usually using.

The Osmoscope web application is a simple single-page app. Only the usual web developer skills around HTML, CSS, and Javascript are needed to understand and change it. It uses OpenLayers for the map display and D3 to show statistics. The rest is just straight Javascript, no fancy Javascript frameworks are needed. Of course the whole thing is Open Source and I encourage every web developer to help out with this. I am not the greatest Javascript developer in the world and the code looks, ah, rather unstructured. So some help there would be appreciated.

So where do these layers come from? That’s not the job of Osmoscope. If you want to create a layer compatible with Osmoscope you have to either create a GeoJSON file available somewhere on the web or set up a vector tile server. If this becomes popular we can probably set up some kind of public server where people can host their vector tile sets or so. But there are plenty of options to get things onto the web. For a GeoJSON file it could be as simple as uploading it to Github. A vector tile server can be something fancy with a database behind it. But for more static data you can simply host the vector tiles as static files. In any case you have to write a simple JSON file containing the metadata of your layer, such as name, URL of the GeoJSON file or vector tiles, description, etc. If you have several layers that belong together you can write a “data source” file, another JSON file that ties them together. Once the URL of the layer file or the data source file is added to Osmoscope, it can figure out the rest from there and display the layer(s).

GeoJSON files are simple to create and, if you want to create a layer for Osmoscope, I would recommend you start out with that. But there is only one file with the data for the whole planet, so if you layer has more than a few thousand objects, it will get slow to use. In that case you have to switch to vector tiles which can cope with basically any number of features. One reasonably simple way of creating those is with Tippecanoe. Tippecanoe can read GeoJSON and create a hierarchy of vector tiles from it.

This is all rather new and I haven’t had time yet to write up the details. And they might change anyway. So for the moment you have to experiment a bit. If you run into any problems, open an issue on Github.