ogr2osm 1.1.1

A tool for converting ogr-readable files like shapefiles into .osm or .pbf data

ogr2osm

A tool for converting ogr-readable files like shapefiles into .pbf or .osm data

Installation

Ogr2osm requires python 3, gdal with python bindings, lxml and optionally protobuf if you want to generate pbf files. Depending on the file formats you want to read you may have to compile gdal yourself but there should be no issues with shapefiles.

Using pip

pip install --upgrade ogr2osm

From source

Clone this repository and run the following command in the created directory.

python setup.py install

Upgrading

If you are upgrading from pnorman's version and you use a translation file for your data, be sure to read about the modifications you will need to do.

About

This program is based on pnorman's version of ogr2osm, but is rewritten to make it useable as a general purpose library.

Ogr2osm will read any data source that ogr can read and handle reprojection for you. It takes a python file to translate external data source tags into OSM tags, allowing you to use complicated logic. If no translation is specified it will use an identity translation, carrying all tags from the source to the .pbf or .osm output.

Import Cautions

Anyone planning an import into OpenStreetMap should read and review the import guidelines located on the wiki. When writing your translation file you should look at other examples and carefully consider each external data source tag to see if it should be converted to an OSM tag.

Usage

Ogr2osm can be used as a standalone application, but you can also use its classes in your own python project.

Standalone

usage: ogr2osm [-h] [--version] [-t TRANSLATION] [--encoding ENCODING]
               [--sql SQLQUERY] [--no-memory-copy] [-e EPSG_CODE]
               [-p PROJ4_STRING] [--gis-order]
               [--rounding-digits ROUNDINGDIGITS]
               [--significant-digits SIGNIFICANTDIGITS]
               [--split-ways MAXNODESPERWAY] [--id ID] [--idfile IDFILE]
               [--saveid SAVEID] [-o OUTPUT] [-f] [--pbf] [--no-upload-false]
               [--never-download] [--never-upload] [--locked] [--add-bounds]
               DATASOURCE

positional arguments:
  DATASOURCE            DATASOURCE can be a file path or a org PostgreSQL
                        connection string such as: "PG:dbname=pdx_bldgs
                        user=emma host=localhost" (including the quotes)

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -t TRANSLATION, --translation TRANSLATION
                        Select the attribute-tags translation method. See the
                        translations/ directory for valid values.
  --encoding ENCODING   Encoding of the source file. If specified, overrides
                        the default of utf-8
  --sql SQLQUERY        SQL query to execute on a PostgreSQL source
  --no-memory-copy      Do not make an in-memory working copy
  -e EPSG_CODE, --epsg EPSG_CODE
                        EPSG code of source file. Do not include the 'EPSG:'
                        prefix. If specified, overrides projection from source
                        metadata if it exists.
  -p PROJ4_STRING, --proj4 PROJ4_STRING
                        PROJ.4 string. If specified, overrides projection from
                        source metadata if it exists.
  --gis-order           Consider the source coordinates to be in traditional
                        GIS order
  --rounding-digits ROUNDINGDIGITS
                        Number of decimal places for rounding when snapping
                        nodes together (default: 7)
  --significant-digits SIGNIFICANTDIGITS
                        Number of decimal places for coordinates to output
                        (default: 9)
  --split-ways MAXNODESPERWAY
                        Split ways with more than the specified number of
                        nodes. Defaults to 1800. Any value below 2 - do not
                        split.
  --id ID               ID to start counting from for the output file.
                        Defaults to 0.
  --idfile IDFILE       Read ID to start counting from from a file.
  --saveid SAVEID       Save last ID after execution to a file.
  -o OUTPUT, --output OUTPUT
                        Set destination .osm file name and location.
  -f, --force           Force overwrite of output file.
  --pbf                 Write the output as a PBF file in stead of an OSM file
  --no-upload-false     Omit upload=false from the completed file to suppress
                        JOSM warnings when uploading.
  --never-download      Prevent JOSM from downloading more data to this file.
  --never-upload        Completely disables all upload commands for this file
                        in JOSM, rather than merely showing a warning before
                        uploading.
  --locked              Prevent any changes to this file in JOSM, such as
                        editing or downloading, and also prevents uploads.
                        Implies upload="never" and download="never".
  --add-bounds          Add boundaries to output file

As a library

Example code:

import logging
import ogr2osm

# 1. Set the logging level of the logger object named 'ogr2osm' to the desired output level

ogr2osmlogger = logging.getLogger('ogr2osm')
ogr2osmlogger.setLevel(logging.ERROR)
ogr2osmlogger.addHandler(logging.StreamHandler())

# 2. Required parameters for this example:

# - datasource_parameter is a variable holding the input filename, or a
#   database connection such as "PG:dbname=pdx_bldgs user=emma host=localhost"
datasource_parameter = ...

# - in case your datasource is a database, you will need a query
query = ...

# - the output file to write
output_file = ...

# 3. Create the translation object. If no translation is required you
#    can use the base class from ogr2osm, otherwise you need to instantiate
#    a subclass of ogr2osm.TranslationBase
translation_object = ogr2osm.TranslationBase()

# 4. Create the ogr datasource. You can specify a source projection but
#    EPSG:4326 will be assumed if none is given and if the projection of the
#    datasource is unknown.
datasource = ogr2osm.OgrDatasource(translation_object)
datasource.open_datasource(datasource_parameter)

# 5. If the datasource is a database then you must set the query to use.
#    Setting the query for any other datasource is useless but not an error.
datasource.set_query(query)

# 6. Instantiate the ogr to osm converter class ogr2osm.OsmData and start the
#    conversion process
osmdata = ogr2osm.OsmData(translation_object)
osmdata.process(datasource)

# 7. Instantiate either ogr2osm.OsmDataWriter or ogr2osm.PbfDataWriter and
#    invoke output() to write the output file. If required you can write a
#    custom datawriter class by subclassing ogr2osm.DataWriterBase.
datawriter = ogr2osm.OsmDataWriter(output_file)
osmdata.output(datawriter)

Refer to contour-osm for a complete example with a custom translation class and coordinate reprojection.

Translations

ogr2osm supports custom translations for your data. To do this you need to subclass ogr2osm.TranslationBase and override the methods in which you want to run custom code.

class TranslationBase:
    def filter_layer(self, layer):
        '''
        Override this method if you want to modify the given layer,
        or return None if you want to suppress the layer
        '''
        return layer


    def filter_feature(self, ogrfeature, layer_fields, reproject):
        '''
        Override this method if you want to modify the given feature,
        or return None if you want to suppress the feature
        Note 1: layer_fields contains a tuple (index, field_name, field_type)
        Note 2: reproject is a function to convert the feature to 4326 projection
        with coordinates in traditional gis order. However, do not return the
        reprojected feature since it will be done again in ogr2osm.
        '''
        return ogrfeature


    def filter_tags(self, tags):
        '''
        Override this method if you want to modify or add tags to the xml output
        '''
        return tags


    def merge_tags(self, geometry_type, tags_existing_geometry, tags_new_geometry):
        '''
        This method is called when two geometries are found to be duplicates.
        Override this method if you want to customize how the tags of both
        geometries should be merged. The parameter geometry_type is a string
        containing either 'node', 'way', 'reverse_way' or 'relation', depending
        on which type of geometry the tags belong to. Type 'reverse_way' is a
        special case of 'way', it indicates both ways are duplicates when one
        of them is reversed. The parameter tags_existing_geometry is a
        dictionary containing a list of values for each key, the list will be
        concatenated to a comma-separated string when writing the output file.
        The parameter tags_new_geometry is a dictionary containing a string
        value for each key.
        Return None if the tags cannot be merged. As a result both geometries
        will be included in the output file, each with their respective tags.
        Warning: not merging geometries will lead to invalid osm files and
        has an impact on the detection of duplicates among their parents.
        '''
        tags = {}
        # ...
        # Default behaviour: add all tags from both geometries
        # If both contain the same key with a different value, then relate
        # the key to a comma separated list of both values
        # ...
        return tags


    def get_unique_node_identifier(self, rounded_x, rounded_y, tags):
        '''
        DEPRECATED -- USE merge_tags IF YOU WANT TO DISABLE MERGING!
        '''
        return (rounded_x, rounded_y)


    def process_feature_post(self, osmgeometry, ogrfeature, ogrgeometry):
        '''
        This method is called after the creation of an OsmGeometry object. The
        ogr feature and ogr geometry used to create the object are passed as
        well. Note that any return values will be discarded by ogr2osm.
        '''
        pass


    def process_output(self, osmnodes, osmways, osmrelations):
        '''
        Override this method if you want to modify the list of nodes, ways or
        relations, or take any additional actions right before writing the
        objects to the OSM file. Note that any return values will be discarded
        by ogr2osm.
        '''
        pass