filtering of OSM pbf-files
Project description
esy-osmfilter
esy-osmfilter
is a Python library to read and filter OpenStreetMap data files in the Protocol
Buffers (PBF) format and export them to a Python dictionary and/or JSON file.
Install
foo@bar:~$ pip install esy-osmfilter
Usage
In the following example the prefilter of esy-osmfilter is used to extract all pipelines from Lichtenstein.
First, all necessary libraries and functions are imported.
>>> import configparser, contextlib
>>> import os, sys
>>> from esy.osmfilter import osm_colors as CC
>>> from esy.osmfilter import run_filter
>>> from esy.osmfilter import Node, Way, Relation
Thereafter, the IO filepaths are defined, assuming the osm-pbf file is already downloaded.
>>> PBF_inputfile = os.path.join(os.getcwd(),
... 'tests/input/liechtenstein-191101.osm.pbf')
>>> JSON_outputfile = os.path.join(os.getcwd(),
... 'tests/output/LI/liechtenstein-191101.json')
Alternatively, you could also make use of urllib library to retrieve a OSM file:
>>> import urllib.request
>>> if not os.path.exists('tests/input/liechtenstein-191101.osm.pbf'):
... filename, headers = urllib.request.urlretrieve(
... 'https://download.geofabrik.de/europe/liechtenstein-191101.osm.pbf',
... filename='lichtenstein-191101.osm.pbf'
... )
... PBF_inputfile = filename
In the next step, a prefilter for all pipeline objects is defined. With the prefilter, we accept all way-items that have "man_made" as key and "pipeline" as value in their taglist. The white and black filter are left empty for the moment.
>>> prefilter = {Node: {}, Way: {"man_made":["pipeline",],}, Relation: {}}
>>> whitefilter = []
>>> blackfilter = []
The run_filter function will allow to filter for OSM items from a pbf-file. We confirm the prefilter phase by setting the boolean variable NewPreFilterData=True.
>>> [Data,_]=run_filter('noname',
... PBF_inputfile,
... JSON_outputfile,
... prefilter,
... whitefilter,
... blackfilter,
... NewPreFilterData=True,
... CreateElements=False,
... LoadElements=False,
... verbose=True)
The prefilter returns the filter results to the Data
dictionary.
This means all OSM way-items with the tag "man_made"="pipeline" are stored there.
But not enough, additionally, all referenced node items of these pipelines are stored there too.
>>> len(Data['Node'])
13
>>> len(Data['Relation'])
0
>>> len(Data['Way'])
2
In this example, we have only found two pipelines and their correspondent 13 nodes.
PLEASE NOTICE:
You can also set "man_made":True to accept items independently of a key value.
In the next step we use run_filter to load the Data
dictionary and specify the main filtering results.
In this example, we use the blackfilter to exclude possible pipelines substations from our prefiltering results.
>>> blackfilter = [("pipeline","substation"),]
We further only accept the drain pipelines that have the really great name "Wäschgräbli".
>>> whitefilter =[(("waterway","drain"),("name","Wäschgräbli")), ]
We initiate the mainfilter phase by setting CreateElements=True.
>>> [_,Elements]=run_filter('funny-waterway-pipelines',
... PBF_inputfile,
... JSON_outputfile,
... prefilter,
... whitefilter,
... blackfilter,
... NewPreFilterData=False,
... CreateElements=True,
... LoadElements=False,
... verbose=True)
>>> len(Elements['funny-waterway-pipelines']['Node'])
0
>>> len(Elements['funny-waterway-pipelines']['Relation'])
0
>>> len(Elements['funny-waterway-pipelines']['Way'])
1
We see, that there is only one way-item left in the Elements
dictionary, the other has been filtered out.
There are no referenced nodes (or relation members) of the remaining way-item passed to the Elements
dictionary.
However, these are still accessible in the Data
dictionary, as the following example shows.
>>> elementlist=list(Elements['funny-waterway-pipelines']['Way'].keys())
>>> first_element=elementlist.pop()
>>> first_item_refs=list(Elements['funny-waterway-pipelines']['Way'][first_element]['refs'])
>>> first_ref_id=str(first_item_refs[0])
>>> first_ref_item=Data['Node'][first_ref_id]
>>> first_ref_item['lonlat']
[9.5022336, 47.15342740000002]
We see that the remaining pipeline has 3 node references.
The first referenced node of the first element was selected from the Data
dictionary via its id-string.
>>> from esy.osmfilter import export_geojson
>>> export_geojson(Elements['funny-waterway-pipelines']['Way'],Data,filename='test.geojson',jsontype='Line')
[[9.5022336, 47.15342740000002], [9.502300000000012, 47.15356290000002], [9.502246300000012, 47.15384200000002]]
{'name': 'Wäschgräbli', 'location': 'underground', 'man_made': 'pipeline', 'waterway': 'drain'}
For more details, jump to the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for esy_osmfilter-1.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2f441fa5ea9282c951ef983759d7750163c72e91e10cfd57c83c8bf1d32977c9 |
|
MD5 | d4cdcda16de10c7488786c5874cf47bd |
|
BLAKE2b-256 | c1d1e56b1d555a43add634fca9bc0ec649cac91dde9a978e61948f2a004f52a5 |