Filtering of OSM pbf-files and exporting to geojson
Project description
esy-osmfilter
esy-osmfilter
is a Python library to read and filter OpenStreetMap data files in the Protocol
Buffers (PBF) format and export them to a Python dictionary and/or JSON file.
Install
foo@bar:~$ pip install esy-osmfilter
Usage
In the following example the prefilter of esy-osmfilter is used to extract all pipelines from Liechtenstein. The easiest way to run esy-osmfilter is to download the sample.py from https://gitlab.com/dlr-ve-esy/esy-osmfilter and run it. It is quite similar to the more detailed desription below.
In this example, we start by importing all necessary libraries and methods.
>>> import configparser, contextlib
>>> import os, sys
>>> from esy.osmfilter import osm_colors as CC
>>> from esy.osmfilter import run_filter
>>> from esy.osmfilter import Node, Way, Relation
Thereafter, the IO filepaths are defined, assuming the osm-pbf file is already downloaded.
>>> PBF_inputfile = os.path.join(os.getcwd(),
... 'tests/input/liechtenstein-191101.osm.pbf')
>>> JSON_outputfile = os.path.join(os.getcwd(),
... 'tests/output/LI/liechtenstein-191101.json')
Alternatively, you could also make use of urllib library to retrieve a OSM file:
>>> import urllib.request
>>> if not os.path.exists('tests/input/liechtenstein-191101.osm.pbf'):
... filename, headers = urllib.request.urlretrieve(
... 'https://download.geofabrik.de/europe/liechtenstein-191101.osm.pbf',
... filename='liechtenstein-191101.osm.pbf'
... )
... PBF_inputfile = filename
In the next step, a prefilter for all pipeline objects is defined. With the prefilter, we accept all way-items that have "man_made" as key and "pipeline" as value in their taglist. The white and black filter are left empty for the moment.
>>> prefilter = {Node: {}, Way: {"man_made":["pipeline",],}, Relation: {}}
>>> whitefilter = []
>>> blackfilter = []
The run_filter function will allow to filter for OSM items from a pbf-file. We confirm the prefilter phase by setting the boolean variable NewPreFilterData=True.
>>> [Data,_]=run_filter('noname',
... PBF_inputfile,
... JSON_outputfile,
... prefilter,
... whitefilter,
... blackfilter,
... NewPreFilterData=True,
... CreateElements=False,
... LoadElements=False,
... verbose=True)
The prefilter returns the filter results to the Data
dictionary.
This means all OSM way-items with the tag "man_made"="pipeline" are stored there.
But not enough, additionally, all referenced node items of these pipelines are stored there too.
>>> len(Data['Node'])
13
>>> len(Data['Relation'])
0
>>> len(Data['Way'])
2
In this example, we have only found two pipelines and their correspondent 13 nodes.
PLEASE NOTICE:
You can also set "man_made":True to accept items independently of a key value.
In the next step we use run_filter to load the Data
dictionary and specify the main filtering results.
In this example, we use the blackfilter to exclude possible pipelines substations from our prefiltering results.
>>> blackfilter = [("pipeline","substation"),]
We further only accept the drain pipelines that have the really great name "Wäschgräbli".
>>> whitefilter =[(("waterway","drain"),("name","Wäschgräble")), ]
We initiate the mainfilter phase by setting CreateElements=True.
>>> [_,Elements]=run_filter('funny-waterway-pipelines',
... PBF_inputfile,
... JSON_outputfile,
... prefilter,
... whitefilter,
... blackfilter,
... NewPreFilterData=False,
... CreateElements=True,
... LoadElements=False,
... verbose=True)
>>> len(Elements['funny-waterway-pipelines']['Node'])
0
>>> len(Elements['funny-waterway-pipelines']['Relation'])
0
>>> len(Elements['funny-waterway-pipelines']['Way'])
1
We see, that there is only one way-item left in the Elements
dictionary, the other has been filtered out.
There are no referenced nodes (or relation members) of the remaining way-item passed to the Elements
dictionary.
However, these are still accessible in the Data
dictionary.
Esy-osmfilter comes with an export function for GeoJSON files (not implemented for relations yet) which will make thinks a lot easier:
>>> from esy.osmfilter import export_geojson
>>> export_geojson(Elements['funny-waterway-pipelines']['Way'],Data,
... filename='test.geojson',jsontype='Line')
To visualize the output-file just open http://geojson.io and drag it on the screen.
For more details, jump to the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file esy-osmfilter-1.0.11.tar.gz
.
File metadata
- Download URL: esy-osmfilter-1.0.11.tar.gz
- Upload date:
- Size: 17.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.7.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
97a0aef58fb34822ef1309942bbeb31170cae10f7d61e245cecffe3d2a467ddd
|
|
MD5 |
da41eb452b3a8cb52f34cb43f84c7560
|
|
BLAKE2b-256 |
c18396107674bdef29fb3d403e730d79e5bd2bdf201594ba35cdb8985eefb178
|
File details
Details for the file esy_osmfilter-1.0.11-py3-none-any.whl
.
File metadata
- Download URL: esy_osmfilter-1.0.11-py3-none-any.whl
- Upload date:
- Size: 30.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.7.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
877c19edcf7340caf2b4cea776ceea008a8fa3a1bdd1966c2bf7642ed82af314
|
|
MD5 |
055cb81d024001480dc9f2cd23ee866a
|
|
BLAKE2b-256 |
edfea4c19693261c9d8e37d73d4cfdc668b5aca78a95daaa80c29da8e45501de
|