Difference between revisions of "Networks/Import/OpenStreetMap"
|Line 286:||Line 286:|
* if your network is larger than ~500MB make sure you use the 64bit version of sumo and use a computer with plenty of RAM
* if your network is larger than ~500MB make sure you use the 64bit version of sumo and use a computer with plenty of RAM
Revision as of 08:15, 9 July 2018
"OpenStreetMap is a free editable map of the whole world. It is made by people like you." (from http://www.openstreetmap.org). This page discusses the conversion of files with data from OpenStreetMap to a SUMO network file.
There are several ways how to download the data from OpenStreetMap to a file. Please read the page Networks/Import/OpenStreetMapDownload to learn about these ways. For more information about the file format visit the page OpenStreetMap file.
- 1 3-click scenario generation
- 2 Importing the Road Network
- 3 Importing additional Polygons (Buildings, Water, etc.)
- 4 Import Scripts
- 5 Elevation Data
- 6 Further Notes
- 6.1 Junctions
- 6.2 Traffic Lights
- 6.3 Highway On- and Off-Ramps
- 6.4 Roundabouts
- 6.5 Isolated Edges
- 6.6 Editing OSM networks
- 7 NETCONVERT Details
- 8 Importing large Networks
- 9 Importing Public Transport Data
- 10 Missing Descriptions
- 11 References
3-click scenario generation
By using the server.py script, a complete scenario can be built quickly and comfortably. The network will be imported with options and typemaps suitable for the selected traffic modes. If more control is needed the options discussed below can be used.
Importing the Road Network
The following call to NETCONVERT imports the road network stored in "berlin.osm.xml" and stores the SUMO-network generated from this data into "berlin.net.xml":
netconvert --osm-files berlin.osm.xml -o berlin.net.xml
OSM-data has always WGS84 geo coordinates which will be automatically UTM transformed by netconvert (since sumo 0.11.1). Thus you need explicit projection parameters only if you need a different projection. Refer to the NETCONVERT documentation for other conversion options.
The number of tiles given in both calls must match.
Recommended NETCONVERT Options
--geometry.remove --roundabouts.guess --ramps.guess \ --junctions.join --tls.guess-signals --tls.discard-simple --tls.join
- --geometry.remove: Simplifies the network (saving space) without changing topology
- --roundabouts.guess: This sets the appropriate right-of-way rules at roundabouts. (Explicit right-of-way rules are not imported from OSM). If this option is not used and roundabouts are not defined manually, then traffic jams will likely occur at roundabouts
- --ramps.guess. Acceleration/Deceleration lanes are often not included in OSM data. This option identifies likely roads that have these additional lanes and causes them to be added
- --junctions.join. See #Junctions
- --tls.guess-signals, --tls.discard-simple, --tls.join. See #Traffic_Lights
Countries with left-hand driving
There are multiple degrees of freedom when importing data from OSM. Sometimes, information such as speed limit is missing in the raw data and must be inferred from the abstract type of the road (i.e. motorway). Different simulation scenarios require different modes of traffic and thus different parts of the traffic infrastructure to be imported. The tool for making these choices are via typemaps. SUMO provides recommended typemaps in the folder <SUMO_HOME>/data/typemap/. They are explained below.
- osmNetconvert.typ.xml default settings. appropriate for rural and motorway scenarios. This is used in the absence of user-specified types. All other typemaps are intended as patches to this typemap
- osmNetconvertUrbanDe.typ.xml Changes default speeds to reflect typical urban speed limits (50km/h)
- osmNetconvertPedestrians.typ.xml Adds sidewalks for some edge types and sets permissions appropriate for pedestrian simulation
- osmNetconvertBicycle.typ.xml imports bicycle lanes
- osmNetconvertShips.typ.xml Imports waterways and ferry routes. This typemap can be combined with any other typemap.
- osmBidiRailNetconvert.typ.xml. Changes the default from uni-directional railroads to bi-directional railroads. This may be usefull in some regions of the world where OSM contributors used this style of date representation. The use of this typemap supplants the older option --osm.railway.oneway-default <BOOL>.
Importing additional Polygons (Buildings, Water, etc.)
OSM-data not only contains the road network but also a wide range of additional polygons such as buildings and rivers.
These polygons can be imported using POLYCONVERT and then added to a
To interpret the OSM-data an additional typemap-file is required (the example below is identical to <SUMO_HOME>/data/typemap/osmPolyconvert.typ.xml):
<polygonTypes> <polygonType id="waterway" name="water" color=".71,.82,.82" layer="-4"/> <polygonType id="natural" name="natural" color=".55,.77,.42" layer="-4"/> <polygonType id="natural.water" name="water" color=".71,.82,.82" layer="-4"/> <polygonType id="natural.wetland" name="water" color=".71,.82,.82" layer="-4"/> <polygonType id="natural.wood" name="forest" color=".55,.77,.42" layer="-4"/> <polygonType id="natural.land" name="land" color=".98,.87,.46" layer="-4"/> <polygonType id="landuse" name="landuse" color=".76,.76,.51" layer="-3"/> <polygonType id="landuse.forest" name="forest" color=".55,.77,.42" layer="-3"/> <polygonType id="landuse.park" name="park" color=".81,.96,.79" layer="-3"/> <polygonType id="landuse.residential" name="residential" color=".92,.92,.89" layer="-3"/> <polygonType id="landuse.commercial" name="commercial" color=".82,.82,.80" layer="-3"/> <polygonType id="landuse.industrial" name="industrial" color=".82,.82,.80" layer="-3"/> <polygonType id="landuse.military" name="military" color=".60,.60,.36" layer="-3"/> <polygonType id="landuse.farm" name="farm" color=".95,.95,.80" layer="-3"/> <polygonType id="landuse.greenfield" name="farm" color=".95,.95,.80" layer="-3"/> <polygonType id="landuse.village_green" name="farm" color=".95,.95,.80" layer="-3"/> <polygonType id="tourism" name="tourism" color=".81,.96,.79" layer="-2"/> <polygonType id="military" name="military" color=".60,.60,.36" layer="-2"/> <polygonType id="sport" name="sport" color=".31,.90,.49" layer="-2"/> <polygonType id="leisure" name="leisure" color=".81,.96,.79" layer="-2"/> <polygonType id="leisure.park" name="tourism" color=".81,.96,.79" layer="-2"/> <polygonType id="aeroway" name="aeroway" color=".50,.50,.50" layer="-2"/> <polygonType id="aerialway" name="aerialway" color=".20,.20,.20" layer="-2"/> <polygonType id="shop" name="shop" color=".93,.78,1.0" layer="-1"/> <polygonType id="historic" name="historic" color=".50,1.0,.50" layer="-1"/> <polygonType id="man_made" name="building" color="1.0,.90,.90" layer="-1"/> <polygonType id="building" name="building" color="1.0,.90,.90" layer="-1"/> <polygonType id="amenity" name="amenity" color=".93,.78,.78" layer="-1"/> <polygonType id="amenity.parking" name="parking" color=".72,.72,.70" layer="-1"/> <polygonType id="power" name="power" color=".10,.10,.30" layer="-1" discard="true"/> <polygonType id="highway" name="highway" color=".10,.10,.10" layer="-1" discard="true"/> <polygonType id="boundary" name="boundary" color="1.0,.33,.33" layer="0" fill="false" discard="true"/> <polygonType id="admin_level" name="admin_level" color="1.0,.33,.33" layer="0" fill="false" discard="true"/> </polygonTypes>
Using the typemap file typemap.xml the following call to POLYCONVERT imports polygons from OSM-data and produces a Sumo-polygon file.
polyconvert --net-file berlin.net.xml --osm-files berlin.osm --type-file typemap.xml -o berlin.poly.xml
The created polygon file berlin.poly.xml can then be added to a
<configuration> <input> <net-file value="berlin.net.xml"/> <additional-files value="berlin.poly.xml"/> </input> </configuration>
The help script osmGet.py allows downloading a large area. The resulting file called "<PREFIX>.osm.xml" can then be imported using the script osmBuild.Py. Both scripts are located in <SUMO_HOME>/tools/import/osm.
The call is:
osmGet.py --bbox <BOUNDING_BOX> --prefix <NAME> osmBuild.py --osm-file <NAME>.osm.xml [--vehicle-classes (all|road|passenger)] [--type-file <TYPEMAP_FILE>] [--netconvert-options <OPT1,OPT2,OPT3>] [--polyconvert-options <OPT1,OPT2,OPT3>]
If "road" is given as parameter, only roads usable by road vehicles are extracted, if "passenger" is given, only those accessible by passenger vehicles.
When using the option --type-file an additional output file with polygons of rivers and buildings as well as Points of Interest (POIs) will be generated. This can be loaded in SUMO-GUI for additional visualization. Useful type files can be found at <SUMO_HOME>/data/typemap/.
Note that the scripts also support a secondary syntax for loading even large areas by splitting them into multiple tiles and download requests. In this case the calls look like this:
osmGet.py --bbox <BOUNDING_BOX> --prefix <NAME> --oldapi --tiles <INT> osmBuild.py --oldapi-prefix <NAME --tiles <INT> [--vehicle-classes (all|road|passenger),ramps,tls] [--type-file <TYEPMAP_FILE>]
Incooperating z-coordinates in networks is still experimental so please report anything odd.
When using option --osm.elevation, z-data is imported from tags with key="ele" in OSM nodes. Since this tag is not yet in wide use, tools exist to overlay OSM data with elevation data sources (http://wiki.openstreetmap.org/wiki/Srtm_to_Nodes). When using the osmosis-srtm pluging the option tagName=ele must be used since only the 'ele' tag is evaluated and the plugin would use the 'height' tag by default.
When using option --osm.layer-elevation, z-data is imported from tags with key="layer" in OSM ways. Since this data does not encode full elevation information, a heuristic is used to interpret the given information. Manual correction may be necessary.
Other data sources
Further options for importing elevation data are listed at the Elevation overview page.
In OpenStreetMap roads forming a single street and separated by, for example, a lawn or tram line, are represented by two edges that are parallel to each other. When crossing with another street, they form two junctions instead of one. To merge such junctions into a single junction, one can define which nodes to merge. See Networks/Building Networks from own XML-descriptions#Joining Nodes and NETCONVERT documentation for usage details.
The NETCONVERT option --junctions.join applies a heuristic to join these junction clusters automatically and is used by default when using the osmBuild.py script described above. However, some junction clusters are too complex for the heuristic and should be checked manually (as indicated by the warning messages). To manually specify joins for these junctions, see JoiningNodes Also, sometimes the heuristic wrongly joins some junctions. These can be excluded by giving them as a list to the option --junctions.join-exclude <ID>[,<ID>]*.
When leaving junctions unjoined, there is a high risk of getting low throughput, jams and even deadlocks due to the short intermediate edges and the difficulty in computing proper traffic light plans for the junction clusters.
Interpreting traffic light information in OSM
NETCONVERT prefers each intersection to be represented by a single node with a single traffic light controller. To achieve the former, see #Junctions. To achieve the latter some extra options are recommended. OSM often uses nodes ahead of an intersection to represent the position of traffic light signals. The actual intersection itself is then not marked as controlled. To interpret these structures the option --tls.guess-signals and --tls.guess-signals.dist <FLOAT> may be used. To cover the cases where this heuristic fails, the options below may be used to computed a joint tls plan for multiple nodes.
Joining traffic lights
OSM does not have the possibility to assign several nodes to a single traffic light. This means that near-by nodes, normally controlled by one traffic light system are controlled by two after the network is imported. It is obvious that traffic collapses in such areas if both traffic lights are not synchronized. Better representation of the reality can often be achieved by joining nearby junctions into a single junction. However, if the junctions should stay separate, it is possible to at least generate a joint controller by setting the option --tls.join. For fine-tuning of joint traffic lights, the attribute tl can be customized for individual nodes.
Debugging missing traffic lights
Occasionally intersections that should be TLS-controlled are set to uncontrolled in the exported .net.xml-file. This may either be due to lack of data in OSM, or due to the invalid interpretation of that data by NETCONVERT. Either of the following steps may be useful to diagnose the problem:
- run NETCONVERT without the options --tls.discard-loaded --tls.discard-simple
- run POLYCONVERT with a type-file that contains <polygonType id="highway" name="highway" and then look at the generated POIs in SUMO-GUI. They should include all traffic light locations defined in the OSM file.
Overriding the traffic light information
If the traffic light information embedded in the OSM file does not fit your needs, you can strip it with --osm.discard-tls option in NETCONVERT and then provide your own definition in a separate *.nod.xml file in a second run of NETCONVERT:
# 1. Import the OSM file to SUMO, discarding TLS information. netconvert --osm-files berlin.osm.xml --output-file berlin-without-tls.net.xml \ --osm.discard-tls # 2. Set traffic light information. netconvert --sumo-net-file berlin-without-tls.net.xml --node-files tls-controlled-nodes.nod.xml \ --output-file berlin-with-tls.net.xml
where tls-controlled-nodes.nod.xml overwrites the type of node to "traffic_light". If the node already exists (which is usually the case) you don't have to provide any information other than the node's ID and new node type.
Highway On- and Off-Ramps
OSM networks often lack additional lanes for highway on- and off-ramps. They can be guessed via NETCONVERT using the --guess-ramps option.
To ensure correct right-of-way at roundabouts, the option --roundabouts.guess should be added. This option is set automatically when using the osmBuild.py script.
When dealing with strictly vehicular scenarios it usually helps to add the option
To discard edges which have no predecessor and no successor edge. However, this often causes the removal of railways or waterways which is not desirable for multi-modal scenarios.
Editing OSM networks
From George Dita, on 01.07.2009 JOSM can be used to edit OSM-data (i.e. for trimming a rectangular map and deleting unwated features). After you delete the part that does not interest you have to alter the file using xmlstarlet which actually deletes the nodes.
xmlstarlet can be used like this:
xmlstarlet ed -d "/osm/*[@action='delete']" < input.osm > output.osm
From Christian Klotz, on 01.07.2009, tip by Christoph Sommmer
The java tool osmosis (http://wiki.openstreetmap.org/index.php/Osmosis) can be used to filter out unwanted features from an OSM-file. The following command keeps motorways and motorway links while filtering out everything else:
java -jar osmosis.jar --read-xml file="orginal.osm.xml" --way-key-value \ keyValueList="highway.motorway,highway.motorway_link" \ --used-node --write-xml file="filtered.osm.xml"
When importing road networks, NETCONVERT searches for the street type, encoded in OSM as a key/value-pair where the key is either "highway", "railway" or "waterway". Only if such a key occures in the edge definition, the edge is imported (see also below). The edge's type name is built from the found key/value pair by building a name as: <KEY>.<VALUE>. Using this type name, the edge's attributes are determined using a predefined map of type names to type definitions. It is possible to override the default types with own type definitions. This is documented in the article about the SUMO edge type file.
If no explicit type file is given, the default one located at SUMO_HOME/data/typemap/osmNetconvert.typ.xml is used. If you want to change the values to add pedestrian infrastructure or have bidirectional railway edges you may want to load additional type maps from SUMO_HOME/data/typemap/.
Explicite Road Attributes
In the case an edge contains the definition about the number of lanes (key="lanes") or the allowed speed (key="maxspeed"), this information is used instead of the according type's value. Also, the per-edge information whether the edge is a one-way edge is read (key="oneway"). In the case the edge belongs to a roundabout (key="junction" and value="roundabout"), it is also set as being a one-way edge.
Dismissing unwanted traffic modes
In most cases, tracks and edges which not may be crossed by motorised traffic are not interesting for road traffic research. It is possible to exclude these edges from being imported using the NETCONVERT-option --remove-edges.by-vclass <STRING>[,<STRING>]*.
For removing all edges which can not be used by passenger vehicles the call must be extended by:
--remove-edges.by-vclass hov,taxi,bus,delivery,transport,lightrail,cityrail, \ rail_slow,rail_fast,motorcycle,bicycle,pedestrian
For removing all edges which can not be used by road vehicles the call must be extended by:
Relationship between OSM ids and SUMO-ids
For the most part, the relationship between OSM and .net.xml is simple:
- OSM node id "1234" translates into junction id "1234".
- OSM way id "5677" translates into multiple edge ids "5678#0", "5678#1", "5678#2", "-5678#0", "-5678#1", "-5678#2"
OSM ways are mostly bi-directional and cross multiple intersections. In .net.xml they are split at each intersection with a running index #n and the edges with prefix '-' are those in the opposite direction.
By setting the option --output.original-names, each lane in the network will record the element name from the input network that it is derived from in a child element:
<param key="origID" value="...."/>
Some OSM elements may not appear in the created network because they are joined with other elements or converted to (unnamed) geometry points along an edge (due to opion --geometry-remove)..
Warnings during Import
Several types of warnings and errors with different levels of severity may be issued during OSM import.
|Warning: The referenced geometry information (ref='...') is not known||Unknown osm node references during import.||Can be safely ignored in most cases (unless the user edited the OSM file)|
|Warning: Discarding way '...' because it has only 1 node(s)||Incomplete data in the OSM file (typically at the boundary of the data set).||Can be safely ignored in most cases (unless the user edited the OSM file)|
|Warning: Discarding unusable type "...." (first occurence for edge "....")||Unknown edge types are ignored during import.||Ignore or provide an Edge-type file which contains that type.|
|Warning: Ignoring restriction relation ...||Some data is missing within the OSM file.||Ignore, because this relation most likely falls outside the boundaries of the road network.|
|Warning: Direction of restriction relation could not be determined||Some data is missing within the OSM file.||Ignore, because this relation most likely falls outside the boundaries of the road network.|
Importing large Networks
It is possible to build SUMO-networks for very large areas (i.e. the whole of Scotland) but some precautions should be taken:
- reduce the network size with the following options
--no-internal-links, --keep-edges.by-vclass passenger, --remove-edges.by-type highway.track,highway.services,highway.unsurfaced
- if you only need major roads you can reduce the network further by setting the option
- if your network is larger than ~500MB make sure you use the 64bit version of sumo and use a computer with plenty of RAM
Importing Public Transport Data
Netconvert and further SUMO tools can be used to import public transpot data from OSM. The easiest way to do this is by using the OSM-Web-Wizard tool. Performing the imported without the wizard is explained in Tutorials/PT_from_OpenStreetMap.
The following data is imported:
- Public transport Stops
- Public transport lines that service these stops
Pulic transport schedules which are needed to make use of the above data are generated with a user-defined service period based on a simulation of the lines.
- TLS computation
- computation of lane-2-lane connections
- what is exactly imported (how edge attributes are determined)
- other traffic modes
- Network quality
- http://www.openstreetmap.org/ - the home site
- http://www.openstreetmap.de/ - the German home site
- http://wiki.openstreetmap.org/index.php/Map_Features - information about database attributes