Skip to main content

openplanetdata-airflow

Shared Airflow infrastructure for OpenPlanetData workflows. Provides custom operators, shared constants, and static geospatial data for processing OpenStreetMap data pipelines.

Installation

pip install openplanetdata-airflow

Operators

GolOperator

Runs gol commands inside the OpenPlanetData Docker container, with optional output file redirection.

from openplanetdata.airflow.operators.gol import GolOperator

query = GolOperator(
    task_id="query",
    args=["query", "planet.gol", "a[boundary=administrative]"],
    output_file="/data/output.geojson",
)

Ogr2OgrOperator

Runs ogr2ogr inside a GDAL Docker container. Supports Airflow template rendering and dynamic task mapping with expand().

from openplanetdata.airflow.operators.ogr2ogr import Ogr2OgrOperator

convert = Ogr2OgrOperator(
    task_id="convert",
    args=["-f", "GPKG", "output.gpkg", "input.geojson"],
)

Shared defaults

The openplanetdata.airflow.defaults module provides shared constants:

Constant Description
DOCKER_MOUNT Bind mount configuration for /data
EMAIL_ALERT_RECIPIENTS Alert email recipients
GDAL_FULL_IMAGE GDAL Docker image reference (ubuntu-full)
GDAL_SMALL_IMAGE GDAL Docker image reference (ubuntu-small)
OPENPLANETDATA_IMAGE OpenPlanetData worker Docker image
OPENPLANETDATA_SHARED_DIR Shared data directory path
OPENPLANETDATA_WORK_DIR Working directory path
R2_BUCKET Cloudflare R2 bucket name
R2INDEX_CONNECTION_ID Airflow connection ID for R2

Static data

  • openplanetdata.airflow.data.continents — list of 7 continents with names and slugs
  • openplanetdata.airflow.data.countries — 250 countries and territories with ISO 3166-1 alpha-2 codes and coastline flags

Docker image

The docker/ directory contains a multi-stage Dockerfile that builds an Airflow worker image with geospatial tools pre-installed:

  • aria2 for fast downloads
  • GDAL/OGR for format conversions
  • GOL (Geodesk) for OSM boundary queries
  • osmcoastline for coastline extraction
  • pyosmium for OSM data updates

License

MIT

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

openplanetdata_airflow-1.2.2.tar.gz (9.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

openplanetdata_airflow-1.2.2-py3-none-any.whl (8.4 kB view details)

Uploaded Python 3

File details

Details for the file openplanetdata_airflow-1.2.2.tar.gz.

File metadata

  • Download URL: openplanetdata_airflow-1.2.2.tar.gz
  • Upload date:
  • Size: 9.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for openplanetdata_airflow-1.2.2.tar.gz
Algorithm Hash digest
SHA256 6b3eda5dd57453b474fa5632e7c3376ee3ffdecbd195f028c311fb154838a66f
MD5 10d90f8f8068a9fec4354e070e314629
BLAKE2b-256 2d5c1e94d17e29d3a1d8217417ae63f884ac1d651a020969b0c5065e053afe0e

See more details on using hashes here.

Provenance

The following attestation bundles were made for openplanetdata_airflow-1.2.2.tar.gz:

Publisher: release.yml on openplanetdata/openplanetdata-airflow

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file openplanetdata_airflow-1.2.2-py3-none-any.whl.

File metadata

File hashes

Hashes for openplanetdata_airflow-1.2.2-py3-none-any.whl
Algorithm Hash digest
SHA256 ced5ae528f8470492a26337881a47560bed9c5dd539a9e65c67b9f39fdb5cd67
MD5 70df3788b2bde509dd5ad9a4f381474f
BLAKE2b-256 a10ae22a921dd2feaf4095229b22c7c8cb2f807d214e7e434de96eb49b030ebc

See more details on using hashes here.

Provenance

The following attestation bundles were made for openplanetdata_airflow-1.2.2-py3-none-any.whl:

Publisher: release.yml on openplanetdata/openplanetdata-airflow

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

1.2.2 This release

2 files

1.2.1

2 files

1.2.0

2 files

1.1.0

2 files

1.0.0

2 files

0.2.4

2 files

0.2.3

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.4

2 files

0.1.3

2 files

0.1.2

2 files

0.1.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page