Skip to main content

overture-serverless

PyPI Python versions

Base class for portable, framework-agnostic "job" classes, plus the Airflow-facing backends that launch them. A published package that gets installed at run time wherever a job actually executes, and provides the contract your job subclasses.

It contains:

  • ServerlessPythonJob — abstract base class your jobs subclass (execute_job()), with parameter parsing (get_param()) and logging (log()). Plain Python, no cloud dependencies — runs identically on any backend or on your laptop.
  • backends.fargate.serverless_python_task_group — an Airflow TaskGroup factory that runs a job on AWS ECS Fargate. Requires the fargate extra (pip install overture-serverless[fargate]), which pulls in apache-airflow and apache-airflow-providers-amazon; the base install stays dependency-free since ServerlessPythonJob runs inside the job container, not inside Airflow.

Writing a job

from overture_serverless.job import ServerlessPythonJob


class CollectionJob(ServerlessPythonJob):
    def execute_job(self) -> None:
        path = self.get_param("input_path")
        self.log(f"Processing {path}")
        # ... your logic

Testing locally

cd packages/overture_serverless
uv run python -c "
from overture_serverless.job import ServerlessPythonJob

class ExampleJob(ServerlessPythonJob):
    def execute_job(self) -> None:
        self.log(self.get_param('input_path'))

ExampleJob().run('{\"input_path\": \"s3://...\"}')
"

Launching a job on Fargate

serverless_python_task_group builds an Airflow TaskGroup that provisions an ECS Fargate task, runs your job's runner container, and tears the task definition down afterward. It has no opinion on your AWS account's VPC layout, IAM roles, or container registry — you resolve those and pass them in:

from overture_serverless.backends.fargate import serverless_python_task_group

collect = serverless_python_task_group(
    group_id="collection_job",
    module_name="overture_addresses.collect",
    class_name="CollectionJob",
    python_packages="overture-addresses",
    task_role_arn=my_resolved_role_arn,  # e.g. via STS in your own DAG code
    network_config=my_ecs_network_config,  # ECS `networkConfiguration` dict
    image_uri=my_resolved_runner_image_uri,  # e.g. from your own ECR-URI builder
    ecs_task_builder_factory=MyEcsTaskBuilder,  # your register/run/teardown builder
)

See the serverless_python_task_group docstring for the full parameter list (sizing presets, CodeArtifact coordinates, retries, etc.).

Publishing

See PACKAGE_VERSIONING.md for how a version bump here turns into a PyPI release.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

overture_serverless-0.1.0.tar.gz (279.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

overture_serverless-0.1.0-py3-none-any.whl (15.0 kB view details)

Uploaded Python 3

File details

Details for the file overture_serverless-0.1.0.tar.gz.

File metadata

  • Download URL: overture_serverless-0.1.0.tar.gz
  • Upload date:
  • Size: 279.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for overture_serverless-0.1.0.tar.gz
Algorithm Hash digest
SHA256 f27456f0a971d6e3be332976963a928061954e72dd853673d30a4514f867471b
MD5 48c4083450c314521570f0e6c42ad85e
BLAKE2b-256 fdd284cd96ba46724af6edca802fe703271f0e2e0f79c204ea081a0a384c4865

See more details on using hashes here.

Provenance

The following attestation bundles were made for overture_serverless-0.1.0.tar.gz:

Publisher: publish_packages.yml on OvertureMaps/overture-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file overture_serverless-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for overture_serverless-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 35c1eaa30025ae0e4e1b9225d90e9bc144adf0c504492eb669b469e033aaad9e
MD5 57c298ec07ae3e8f593d4a0c8ebfac09
BLAKE2b-256 c47a5c924e9823791f62e87c4125e4a7cca10848ddd14a98b38efacc6004df5c

See more details on using hashes here.

Provenance

The following attestation bundles were made for overture_serverless-0.1.0-py3-none-any.whl:

Publisher: publish_packages.yml on OvertureMaps/overture-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 files

0.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page