Skip to main content

Iterate over .osm file and call a provided callback function for each element

Project description

This code loads .osm file and allows to call function on all OSM objects in dataset.

Installation

pip install osm-iterator

It is distributed as an osm_iterator PyPI package.

PyPI version

Usage example

from osm_iterator import osm_iterator
import requests
import os.path

def download_from_overpass(query, output_filepath):
  print(query)
  url = "http://overpass-api.de/api/interpreter"
  r = requests.get(url, params={'data': query})
  result = r.text
  with open(output_filepath, 'w') as file:
      file.write(str(result))

def show_places(element):
    place_tag = element.get_tag_value("place")
    name_tag = element.get_tag_value("name")
    osm_object_url = element.get_link()
    if place_tag != None:
        print(name_tag, "(", place_tag, ") is ", osm_object_url)

filepath = "places_in_Kraków.osm"
query = """
[out:xml][timeout:2500];
area[name='Kraków']->.searchArea;
(
  node["place"](area.searchArea);
  way["place"](area.searchArea);
  relation["place"](area.searchArea);
);
out center;
"""

if os.path.isfile(filepath) == False:
    download_from_overpass(query, filepath)
osm = osm_iterator.Data(filepath)
osm.iterate_over_data(show_places)

Running tests

nosetests3 or python3 -m unittest or python3 tests.py

History

Design explanation: this code has deeply suboptimal handling of pretty much everything. For start, all data is loaded into memory and then duplicated in-memory dataset is created.

As result, attempt to process any large datasets will cause issues due to excessive memory consumption.

This situation is consequence of following facts

  • This code was written during my first attempt to process OSM data using Python
  • API allows (at least in theory) to painlessly switch to real iterator that is not loading all data into memory at once
  • So far this was good enough for my purposes so I had no motivation to spend time on improving something that is not a bottleneck

Though, if someone has good ideas for improvements (especially in form of a working code) - comments and pull requests are welcomed.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for osm-iterator, version 1.3.0
Filename, size File type Python version Upload date Hashes
Filename, size osm_iterator-1.3.0-py3-none-any.whl (4.9 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size osm_iterator-1.3.0.tar.gz (4.0 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page