2014-06-15 08:59 by pjotrp

1 OSM on the Garmin

Create a free Garmin map: for cycling, mountain biking, and hiking

by Pjotr Prins (based on work by Minko wizardofoh and others)

This page contains (Linux/OSX) command line instructions for creating a free map for hiking and cycling for your Garmin device, using free tools all based on the openstreetmap database. Anywhere on the planet. You can even add your own styling and contour lines!

If you take a laptop along with the full country osm.pbf file, a matching contours file (in OSM format), osmosis, mkgmap and the style files, then you can make maps on the fly, wherever you are, even without internet access. I tend to bring along the large files, and make smaller hiking/cycling maps wherever I find myself. Of course, you can also make larger maps, as much as your garmin can hold.

Instructions for the impatient

Only two steps to get a routable cycle map for the Garmin!

Basically reduce a large country map file downloaded from http://www.geofabrik.de/ to a size that the Garmin can hold in one 'tile', using the osmosis tool. As a rule of thumb, the OSM file should be smaller than ~800 Mb. Next convert that file to a Garmin file using a style (kind of a CSS).

For example, using the hiking_styles layout

Shell
# Convert to Garmin file with styles attached, set the environment, e.g.,:
alias mkgmap="/opt/jre1.7.0/bin/java -jar $HOME/Downloads/OSM/mkgmap-r3294/mkgmap.jar"
alias osmosis="/home/wrk/Downloads/OSM/osmosis/bin/osmosis"
git clone https://github.com/pjotrp/OSM.git
cd OSM/
# Pre-check the area size, should be less than 1.0(?)
ruby -e "p (($top-$bottom)*($left-$right)).abs"
# Reduce the map size (map.osm should be smaller than an approx. 800 Mb)
osmosis --read-pbf file=france-latest.osm.pbf \
  --bounding-box top=$top left=$left bottom=$bottom right=$right --write-xml "map.osm"
# Check the map size
ls -lrSh *.osm
mkgmap --gmapsupp --style-file=hiking_styles --overview-mapname=ofm_bnl   --family-id=10010 --product-id=1 --family-name=OpenFietsMap_Benelux   --index --draw-priority=28 map.osm 10010.TYP
ls -lh gmapsupp.img

and you should have a usable gmapsupp.img for your garmin device. The latest version of the single tile script can be found in scripts/latest_create_map.sh

To load it into qtlandkarte select the .tbd file.

Shell
mkgmap --tdbfile gmapsupp.img 

to add contours and POIs, get the srtm as described in a later section. With the file contours.osm you can combine them with

Shell
# Reduce the main map
osmosis --read-pbf file=/export/data/OSM/france-latest.osm.pbf   --bounding-box top=$top left=$left bottom=$bottom right=$right --write-xml "map.osm"
# Reduce the contours file to contours.osm
osmosis --read-xml file=srtm.osm --bounding-box top=$top left=$left bottom=$bottom right=$right --write-xml "contours.osm"
# and combine the two maps map.osm and contours.osm
time mkgmap --max-jobs=3 --gmapsupp --style-file=hiking_styles \
  --add-pois-to-areas  --overview-mapname=ofm_bnl --family-id=10010 \
  --product-id=1 --family-name=Cycle --index --draw-priority=28 \
  map.osm 10010.TYP --family-id=10011 --product-id=1 \
  --family-name=contours --draw-priority=30 --transparent \
  contours.osm 10011.TYP --tdbfile

where the values of top, left, right and bottom are the same as on the export tab of http://www.openstreetmap.org/, when selecting a map. For the osmosis and mkgmap commands, see the aliases below.

These tiles tend to be pretty small (good enough for recreational hikes and cycle sessions). For splitting larger files into tiles that can be recombined for the Garmin, see the information on splitter below.

The long explanation

Basically the routine is to define the square area you want a map of (four coordinates, which can be found on the map). Next fetch the free country data, either by downloading a file, or by querying a server. Then limit the data to your predefined square area. Next use splitter to split a large map into smaller sections (the Garmin needs that). Then combine the sections again into a Garmin img file. In the process you can use specific map styles and add mountain contours. It sounds like a lot, but once you get the hang of it, it should be a reasonably fast process. Most of my struggles go with wanting to change the style of the maps. If you can live with other people's work, it should be easy.

The Open Street Map (OSM) is a 'wiki' type initiative for making maps. The project has really taken off now,with close to feature complete countries (like the Netherlands, Germany, United Kingdom, United States etc). The OSM is also very important for developing countries. The best navigation maps of Africa are appearing on OSM, for example.

For cycling and hiking the mapping effort is paying off, there is nothing that matches the OSM. The Netherlands has almost all important cycle paths, see for example this part of the Netherlands on the open cycle map (OCM). The OCM, is a special rendering of the Open Street Map, it is the same database, and the blue dotted lines are free running cycle paths, the blue lines are cycle routes). Any other map you have of the Netherlands is immediately out of date, as the Dutch are laying down 1000 km of cycle paths annually!

The UK is quickly getting dense with walking paths (see this map, which shows part of the ridge way). People are using and creating useful maps. Once you get the hang of it you can create a free new map in 10 minutes. It is truly amazing.

1.1 The Netherlands Cycle map

Now, what we want to achieve here, is to create a map that we can put on our Garmin (the Garmin 60cx model is still the best one, mostly because it is simple, has colour and runs 15 hours on standard AA batteries, and comes below EUR 200 now, get it with an extra one or two 2Gb micro SD's - latest is that I replaced the 60cx with the Dakota 20). You don't need to worry about the default base map, as we won't use it. After some magic you can create a gmapsupp.img file which you can just copy to your Garmin.

Minko, recumbent world cyclist and wizard ('tovenaar') has created the best rendering of the OCM for Garmin. He calls it the openfietsmap. Fiets is Dutch for ... right, bicycle. Look at his pretty pictures. Also he has the map making procedure on his site, which is more oriented toward Windows users.

If you want the Netherlands and Belgium you can just download his gmapsupp.zip image file (through the download tab on his openfietsmap site). And copy the unzipped edition to your Garmin. Basically you download the 500 Mb gmapsupp.img file on to your computer.

Hook your Garmin up with a USB cable to your computer, on the 60cx you lift a rubber flap. Switch the device on, go to the Main Menu, hit Setup and Interface. Select USB Mass Storage and your Garmin should show up as a disk drive on your computer (on Windows, also on OSX and Linux). Copy the gmapsupp.img file to the Garmin folder on that drive (note it is case sensitive). Disable the device on the computer, and you can restart the Garmin. The memory on the device should be large enough to hold the file. If it is too small switch the microSD to something larger - it sits behind the batteries (if the microSD is larger than 2Gb you need to format it on your computer to 2Gb).

And you should be ready to play.

1.2 Creating other Maps

Obviously it is impossible for Minko and others to create every map for everyone.

Probably the easiest way is to fetch OSM data in the form of prepared files. For example you can download whole countries and regions from Geofabrik.de. These files can be combined and reduced (bounding box) with osmosis. For example combine two files, and write the overlapping bounding box around Lubeck to dump.osm:

Shell
  top=54.173; bottom=53.654; left=10.132; right=10.986
  ./bin/osmosis --read-xml file=mecklenburg-vorpommern.osm file=schleswig-holstein.osm enableDateParsing=no --bounding-box top=$top left=$left bottom=$bottom right=$right --write-xml 

or, with the newer bpf format

Shell
./osmosis-0.41/bin/osmosis --read-pbf file=japan.osm.pbf --bounding-box top=$top left=$left bottom=$bottom right=$right --write-xml "map.osm"

So Osmosis will reduce an existing country file to a set within coordinates.

The alternative route is to create a gmapsupp.img map through the OSM api. This, however, often fails because servers easily get overloaded. As above, first you want to get the border co-ordinates of your map. Use the Open Street Map 'Export' tab and zoom and select the map. Don't make it too large, start with a small one. Once you have a nice view of what you want you can get the co-ordinates by jotting down the values on the left in the four boxes (e.g. Brecon Beacons national park has left=6.2581, bottom=53.3454, right=6.3141, top=53.3748).

Now we want to download all information in that area. For this you need software. I use curl, or wget to fetch the OSM data. Examples

Shell
  curl -L "http://xapi.openstreetmap.org/api/0.6/map?bbox=$left,$bottom,$right,$top" -o data.osm
  curl -L "http://www.informationfreeway.org/api/0.6/map?bbox=$left,$bottom,$right,$top" -o data.osm

or, an example with the filled in coordinates:

Shell
  curl -L "http://www.informationfreeway.org/api/0.6/map?bbox=6.2581,53.3454,6.3141,53.3748" -o data.osm

Which downloads data.osm - which can be a large file if you select a large area. The Brecon Beacons, above, will be about 2.5Mb.

If it is not working you may have the coordinates wrong - the API call can be checked by hitting the Export button on OSM.

1.3 Checking the OSM data

To check the contents you can load this file into an OSM editor, like JOSM or Merkaartor. These are programs written especially for viewing and modifying the open street map!

1.4 Using QlandkarteGT

Use the latest version - you may have to install from source (I am using version 1.7.0). Sometimes the program has issues. If you have trouble displaying an updated map (with contours) try removing the config file (actually, I do this every time)

Shell
rm ~/.config/QLandkarteGT/ -rf

and make sure you see the right version (the About screen). Use the --tbdfile switch of mkgmap

Shell
rm *.img *.tdb *.mdx ; time mkgmap map.osm --tdbfile --index --gmapsupp

and open the .tdb file with qlandkartgt.

1.5 Convert to Garmin

Now we want to convert this data to the Garmin format, a procedure described by Minko. Minko splits the file. A smaller file than the Netherlands does not need to be split, so we can do something like:

Shell
  java -Xms512m -Xmx1024m -ea -jar mkgmap-r1580/mkgmap.jar -c osm_nl.args 10010.TYP

obviously you need a few files, like download mkgmap.jar. Mkgmap is a command line utility that converts OpenStreetMap data into vector maps that can be loaded onto a Garmin GPS device. The other two files are by Minko, so let us simplify first

Shell
  alias mkgmap='java -Xmx1024m -Xms128m -ea -jar /opt/mkgmap-r1543/mkgmap.jar'
  mkgmap --max-jobs=3 data.osm

Generates a file 63240001.img which we can copy to the Garmin as gmapsupp.img (using a maximum of 3 CPU cores and 1Gb of heap space). E.g.

Shell
cp 63240001.img /mnt/Garmin/gmapsupp.img

Stop the USB and restart Garmin.

Note you can also view this file with your Garmin Mapsource software, or the latest qlandkarte tool (for qlandkarte use the mkgmap --tdbfile switch instead of --gmapsupp and select the .tdb file!). It is a good way to validate a map, before trying it on the Garmin (see Qlandkartegt section for more info).

Let's get fancier and try adding more rendering features. Follow the instructions for the OSM cycle map. Basically:

Shell
java -Xmx2000m -jar splitter-r200/splitter.jar ../tokyo.osm
time mkgmap --remove-short-arcs --style-file=cyclemap --route --net -c template.args --max-jobs=4
mkgmap --gmapsupp 6324000*.img

Or use Minko's style. Minko has the files in his scripts section. Fetch 10010.TYP, 10011.TYP - both stylesheet type files, and osm_bnl.args - which are the command line arguments for mkgmap.

Shell
  mkgmap data.osm --family-id=10010 --product-id=1 --family-name=MyMap --index 10010.TYP 

which has changed the rendering. Now we want the nice colour on the Garmin that Minko uses. For this we need the files in the Styles folder. Download these files from Minko's site and zip them up (on Linux the directory option of mkgmap does not appear to work):

Shell
  time mkgmap data.osm --tdbfile --max-jobs=3 --family-id=10010 --product-id=1 --family-name=MyMap --index --style-file=styles.zip  10010.TYP 

1.6 Adding contours

One benefit of the Garmin is that it can show contours. Ideal for cycling and hiking. An exciting free database comes from the Space Shuttle project (!). We can use that contour data with the open streetmap! The tool Srtm2Osm.exe requires .NET (Windows) or mono (Linux) and can be found here.

Shell
alias srtm="mono ~/garmin/Srtm2Osm/Srtm2Osm.exe"
srtm -step 50 -large -bounds1 $bottom $left $top $right

saves to srtm.osm. Now we can combine the maps:

Shell
  alias mkgmap='java -Xmx1500m -ea -jar mkgmap-r1543/mkgmap.jar'
  mkgmap --gmapsupp \
    --family-id=10010 \ 
    --product-id=1 \
    --family-name=Cycle \
    --index \
    --draw-priority=25 \
    --style-file=cycle.zip  10010.TYP \
    data.osm \
    --family-id=10011 \
    --product-id=1 \
    --family-name=contours \
    --draw-priority=28 \
    --transparent \
    srtm.osm

You may add --remove-short-arcs --route --net

Shell
mkgmap --gmapsupp \
  --remove-short-arcs 
  --route \
  --net \
  --family-id=10010 \
  --product-id=1 \
  --family-name=Cycle \
  --index \
  --draw-priority=25 \
  --style-file=cyclemap  \
  10010.typ \
  63240001.img \
  --family-id=10011 \
  --product-id=1 \
  --family-name=contours \
  --draw-priority=28 \
  --transparent \
  ../srtm.img

another example, which worked on my last trips:

This one uses default Garmin styles. The minimalistic form is guaranteed to work!!

Shell
mkgmap --gmapsupp --add-pois-to-areas --index --draw-priority=25 \
  canary_islands.osm.pbf --family-name=contours srtm.osm --draw-priority=1000 --transparent

Trying, this works too (old style cycle map):

Shell
mkgmap --gmapsupp --add-pois-to-areas --overview-mapname=ofm_bnl --family-id=10010 \
  --product-id=1 --family-name=OpenFietsMap_Benelux --index \
  --draw-priority=25 canary_islands.osm.pbf 10010.TYP --family-id=10011 \ 
  --product-id=1 --family-name=OpenFietsMap_hoogtelijnen --draw-priority=30 \
  --transparent srtm.osm 10011.TYP

Let's add a style, hiking_styles is an adaptation of an early cycle map style:

Shell
mkgmap --gmapsupp --style-file=hiking_styles \
  --overview-mapname=ofm_bnl --family-id=10010 --product-id=1 \
  --family-name=OpenFietsMap_Benelux --index --draw-priority=28 \
  canary_islands.osm.pbf 10010.TYP --family-id=10011 --product-id=1 \
  --family-name=OpenFietsMap_hoogtelijnen --draw-priority=30 \
  --transparent srtm.osm 10011.TYP

BTW when copying files to the garmin it pays to do a checksum. I use Pfff, e.g.

Shell
  mount /dev/sdc1 /mnt/garmin/
  cp gmapsupp.img /mnt/garmin/Garmin/
  umount /mnt/garmin/
  mount /dev/sdc1 /mnt/garmin/
  pfff -k 1 /mnt/garmin/Garmin/gmapsupp.img gmapsupp.img
  1101000000200001000000000000:A8CE4C5ECE56BCC260B00EFDC0B0DC6B   /mnt/garmin/Garmin/gmapsupp.img
  1101000000200001000000000000:A8CE4C5ECE56BCC260B00EFDC0B0DC6B   gmapsupp.img
  umount /mnt/garmin/

Note that you only need to make a contours file once! The contours of mountains don't change that much in our life span.

1.7 Splitting large files

The format used for Garmin maps has, in effect, a maximum size, meaning that you have to split an .osm file that contains large well mapped regions into a number of smaller 'tiles'. Splitter does that. You can find it here.

Shell
alias mkgmap='java -Xmx4096m -ea -jar ~/garmin/mkgmap-r1543/mkgmap.jar'
alias splitter="java -Xmx2000m -jar ~/garmin/splitter-r123/splitter.jar"
splitter data.osm

You'll find a number of files named 63240001.osm.gz and so on. Now you need to create an args file, which is basically the same as the arguments you pass to mkgmap (splitter creates this file for you!)

Shell
#---- inject the following in the template.args file generated by splitter
family-id: 10010
product-id: 1
draw-priority:28
series-name: OpenFietsMap
family-name: OpenFietsMap
area-name: OpenFietsMap 
description: Routable cycling map
country-abbr: 
country-name: 
style-file: ../cycle.zip
overview-mapname: 63240001
#---- generate-sea: no-mp,extend-sea-sectors,close-gaps=1000,land-tag=natural=background
tdbfile
latin1
code-page: 1252
location-autofill: 1
ignore-maxspeeds
remove-short-arcs
merge-lines
add-pois-to-areas
road-name-pois: 0x6500
preserve-element-order
keep-going
net
route
index
#---- below add the content of the template.args generated by splitter

and run it with

Shell
mkgmap -c template.args 10010.TYP

You can separately fetch the contours using the coordinates output by splitter

Shell
alias srtm="mono ~/garmin/Srtm2Osm/Srtm2Osm.exe"
srtm -step 50 -large -bounds1 $bottom $left $top $right
mkgmap --mapname=srtm --draw-priority=28 --transparent srtm.osm --tdbfile

Writes srtm.img.

Check the new contour file with qtlandkarte and combine the image files again with

mkgmap --gmapsupp 6*.img srtm.img

old:

Shell
mkgmap --gmapsupp 6*.img \
    --family-id=10011 \
    --product-id=1 \
    --family-name=contours \
    --transparent \
    --keep-going \
    srtm.osm \
    10010.TYP \
    10011.TYP

Shell
mkgmap --gmapsupp 6*.img \
    --family-id=10011 \
    --product-id=1 \
    --family-name=contours \
    --draw-priority=28 \
    --transparent \
    --keep-going \
    srtm.osm \
    10010.TYP \
    10011.TYP

If it complains the OSM file is too large, say larger than ~300 Mb, download a smaller contour file (you can change the -step option of srtm, and maybe it is possible to split contour files, also using splitter using the --mixed option

Shell
java -Xmx2000m -jar splitter-r200/splitter.jar --mixed ../nagano-srtm.osm

See also wiki). Remove the empty image file(s) first, before re-running mkgmap again (!) Otherwise you may get an illegal argument exception.

Shell
#---- Make sure you use version 0.35, as srtm is OSM API 0.5
alias osmosis="~/garmin/osmosis-0.35/bin/osmosis"
#---- This converts versoin 0.5 to version 0.6
osmosis --read-xml-0.5 enableDateParsing=no file="srtm.osm" --migrate --write-xml file="srtm-0.6.osm"
#---- Now we can split on a boundary
osmosis --read-xml enableDateParsing=no file=srtm-0.6.osm     --tee 2 \
--bounding-box right=1.0 --wx 53274338.osm.gz   \
--bounding-box left=1.0 --wx 53274339.osm.gz
mkgmap --gmapsupp 6*.img \
    --family-id=10011 \
    --product-id=1 \
    --family-name=contours \
    --draw-priority=28 \
    --transparent \
    --keep-going \
    5*.osm.gz \
    ../10010.TYP \
    ../10011.TYP

where the value 1.0 should be half-way on the map.

Note again that a contour map you only have to create once.

To install the map, simply copy the single file gmapsupp.img to the Garmin directory on your device.

Some people may want to read the img files with mapsource (on Windows). Possible instructions are here.

1.8 Contributing to the Open Cycle Map and Openstreetmap

To contribute to the Open Cycle Map (OCM) - let's say you found a nice cycle path that is not on the Open Street Map yet - the easiest way is to register with the Open Street Map. Next upload a track from your Garmin (one of those GPX files on your device when it is USB mounted on your computer. When you have uploaded the track to the website you can edit it through the 'GPS Traces' tab. Select 'See your traces' and hit 'edit'. This will bring up an online editor which is fairly easy to use. Check the tutorials for the 'Potlatch' editor.

Even when not editing the OSM, uploading the GPX file with a description matters. Other people may add it to the map, or use it to validate other edits.


wikiTEXer /home/wrk/izip/git/opensource/ruby/wikitexer/bin/VERSION by Pjotr Prins - generated 2014-06-15 08:59 by pjotrp