Build your own BRouter segments files

I recently started to have a deeper look into routing for bikes, on top of OpenStreetMap data and came across BRouter. This article is a follow-up to the first one and will cover how to build your own segments files and add new tags to be extracted from the OSM data for your profiles.

Building BRouter files

BRouter uses its own data format (.rd5 files), split in tiles of 5 x 5 in latitude and longitude. You can download the official tiles from brouter.de but you can also build them yourself from an OSM dump.

OSM dumps are available from planet.osm.org (for the full planet files) or geographical extracts are provided by Geofabrik.

Build the pbfparser

First, there are two file formats available to download OSM data: bzip-ed XML files (very large) and .pbf (Protobuf format) which is much more efficient. If you want to use the latter one, you will have to build the pbfparser (in misc/pbfparser first):

• Download the latest version of Osmosis and unzip it somewhere.
• Copy the lib/default/protobuf-java-*.jar and lib/default/osmosis-osm-binary-*.jar files from the unzipped Osmosis archive to misc/pbfparser/protobuf.jar and misc/pbfparser/osmosis.jar.
• Build brouter-server (as we did in the previous article) and copy brouter-server/target/brouter-server-*-jar-with-dependencies.jar to misc/pbfparser/brouter.jar.
• You can build the pbfparser using, in the misc/pbfparser/ folder,

javac -d . -cp "brouter.jar:protobuf.jar:osmosis.jar" *.java

• Finally, you can build a jar file from these files using

jar cf pbfparser.jar btools/**/*.class

Note: If the jar file is not properly created, everything else will seem to work normally but there will not be any data extracted from the OSM data dump. You can check what is actually inside the built jar file using jar tf pbfparser.jar.

You can now move the files brouter.jar, protobuf.jar, pbfparser.jar and osmosis.jar from the misc/pbfparser folder to the misc/scripts/mapcreation folder.

Run the map creation script

You can now run the misc/scripts/mapcreation/process_pbf_planet.sh script to build the segments.

Here are a few useful tips before running it: * Checkout this PR which improves a lot the script and adds documentation. * The script should be run from the misc/scripts/mapcreation folder. * Have a look at the variables defined at the beginning of the files and overwrite them according to your needs. * The script is meant to be used for the BRouter production setup. You likely want to comment the beginning (downloading of the whole planet file and removal of the previous segments) and the very least (copy of the generated segments to the webserver location).

Note: It is possible that you encounter an error complaining about not being able to run bash^M. You can fix this one by running sed -i -e 's/\r$//' process_pbf_planet.sh.

Beware of the size and requirements to handle the whole planet file, it might be better to start with a regional extract from geofabrik.de.

If you want to have elevation information in the generated segments files, you should download the required SRTM files. Any flavor of the 90m SRTM database should be working, but the one used by the official BRouter segments files are the ones provided by CGIAR. If you are working with rather small geographical extracts, you can download tiles manually using this interface (use the “ArcInfo ASCII” format), instead of having to ask for an access for bulk download of data. There is no need to unzip the downloaded files, the process_pbf_planet.sh script expects a folder with all the ZIP files inside and will manage it.

Note that if you don’t have the SRTM data available, the segments files will still be generated without any issue (but they will miss the elevation data). If you are not sure which SRTM files you have to download, you can run the script once and it will log all the SRTM files it is looking for. Also note that there will not be any error shown if you miss a SRTM file.

Custom lookups.dat

Now that we know how to build our own segments files for BRouter, we can start hacking the lookups.dat file to add new tags that we may need.

A very good introduction to the lookups.dat file (not really easy to read, but is full of useful information) is the profile_developers_guide.txt file. The main points to keep in mind are:

• There are two context, delimited by a line starting with --- context:. The way context applies to OSM ways and the node context defines tag extraction rules for OSM nodes.
• The lookups.dat has a lookupversion (major version) and a minorversion defined at the very beginning. Increasing the minorversion means the change is retro-compatible, increasing the lookupversion means it is not. This is important for the BRouter app for Android but if you just play with lookups.dat and brouter-server, this should not matter.
• The numbers are an indication of the frequency of this tag in Germany. You can just put any number and don’t care about this.
• You can define aliases in this file by putting multiple tags values on the same line, separated by a space. For instance

busway;0000056005 opposite opposite_lane opposite_track

means it will extract the busway tag values opposite, opposite_lane and opposite_track and BRouter will expose all these values as the first value, busway=opposite in this case.

• Aliases can actually be used with any name to have a custom alias (even if it is not a known OSM value). You can use zone:maxspeed;0000001616 20 DE:20 FR:20 for instance to have a uniform output (zone:maxspeed=20) for all the zone with speed limitations of 20, no matter the country, even if there was no tag zone:maxspeed=20 within OSM.
• The lookup table used by BRouter afterwards is based on the position of the tag entry in the lookups.dat. Then, you should add new tags at the end (of the appropriate context) or replace one of the “dummy” tags that separate the real way tags from the relation pseudo tags, to make it as retro-compatible as possible. (See this post on the forum)
• Same when you add a tag value to an alias, you should always put the new value at the end.

Note that once you have built the segments files with your new lookups.dat file and ask a route from brouter-server, it might not output all the newly added tags values (in the JSON output, or in the “Data” tab of brouter-web). This is due to the fact that BRouter only outputs tags which are actually used by the profile. You can set assign processUnusedTags = true in your profile to output all tags for the considered ways, even if they were not used (see the dummy profile which can be used for this purpose).

The last article of the serie will be about writing your own profiles and writing test cases to assist you in this task.