overture-serverless
Base class for portable, framework-agnostic "job" classes, plus the Airflow-facing backends that launch them. A published package that gets installed at run time wherever a job actually executes, and provides the contract your job subclasses.
It contains:
ServerlessPythonJob— abstract base class your jobs subclass (execute_job()), with parameter parsing (get_param()) and logging (log()). Plain Python, no cloud dependencies — runs identically on any backend or on your laptop.backends.fargate.serverless_python_task_group— an AirflowTaskGroupfactory that runs a job on AWS ECS Fargate. Requires thefargateextra (pip install overture-serverless[fargate]), which pulls inapache-airflowandapache-airflow-providers-amazon; the base install stays dependency-free sinceServerlessPythonJobruns inside the job container, not inside Airflow.
Writing a job
from overture_serverless.job import ServerlessPythonJob
class CollectionJob(ServerlessPythonJob):
def execute_job(self) -> None:
path = self.get_param("input_path")
self.log(f"Processing {path}")
# ... your logic
Testing locally
cd packages/overture_serverless
uv run python -c "
from overture_serverless.job import ServerlessPythonJob
class ExampleJob(ServerlessPythonJob):
def execute_job(self) -> None:
self.log(self.get_param('input_path'))
ExampleJob().run('{\"input_path\": \"s3://...\"}')
"
Launching a job on Fargate
serverless_python_task_group builds an Airflow TaskGroup that provisions an ECS Fargate task, runs your job's runner container, and tears the task definition down afterward. It has no opinion on your AWS account's VPC layout, IAM roles, or container registry — you resolve those and pass them in:
from overture_serverless.backends.fargate import serverless_python_task_group
collect = serverless_python_task_group(
group_id="collection_job",
module_name="overture_addresses.collect",
class_name="CollectionJob",
python_packages="overture-addresses",
task_role_arn=my_resolved_role_arn, # e.g. via STS in your own DAG code
network_config=my_ecs_network_config, # ECS `networkConfiguration` dict
image_uri=my_resolved_runner_image_uri, # e.g. from your own ECR-URI builder
ecs_task_builder_factory=MyEcsTaskBuilder, # your register/run/teardown builder
)
See the serverless_python_task_group docstring for the full parameter list (sizing presets, CodeArtifact coordinates, retries, etc.).
Publishing
See PACKAGE_VERSIONING.md for how a version bump here turns into a PyPI release.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file overture_serverless-0.1.0.tar.gz.
File metadata
- Download URL: overture_serverless-0.1.0.tar.gz
- Upload date:
- Size: 279.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
f27456f0a971d6e3be332976963a928061954e72dd853673d30a4514f867471b
|
|
| MD5 |
48c4083450c314521570f0e6c42ad85e
|
|
| BLAKE2b-256 |
fdd284cd96ba46724af6edca802fe703271f0e2e0f79c204ea081a0a384c4865
|
Provenance
The following attestation bundles were made for overture_serverless-0.1.0.tar.gz:
Publisher:
publish_packages.yml on OvertureMaps/overture-core
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
overture_serverless-0.1.0.tar.gz -
Subject digest:
f27456f0a971d6e3be332976963a928061954e72dd853673d30a4514f867471b - Sigstore transparency entry: 2617695146
- Sigstore integration time:
-
Permalink:
OvertureMaps/overture-core@20ad654267068b7e30a9f369ed14bd12e00126d2 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/OvertureMaps
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish_packages.yml@20ad654267068b7e30a9f369ed14bd12e00126d2 -
Trigger Event:
push
-
Statement type:
File details
Details for the file overture_serverless-0.1.0-py3-none-any.whl.
File metadata
- Download URL: overture_serverless-0.1.0-py3-none-any.whl
- Upload date:
- Size: 15.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
35c1eaa30025ae0e4e1b9225d90e9bc144adf0c504492eb669b469e033aaad9e
|
|
| MD5 |
57c298ec07ae3e8f593d4a0c8ebfac09
|
|
| BLAKE2b-256 |
c47a5c924e9823791f62e87c4125e4a7cca10848ddd14a98b38efacc6004df5c
|
Provenance
The following attestation bundles were made for overture_serverless-0.1.0-py3-none-any.whl:
Publisher:
publish_packages.yml on OvertureMaps/overture-core
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
overture_serverless-0.1.0-py3-none-any.whl -
Subject digest:
35c1eaa30025ae0e4e1b9225d90e9bc144adf0c504492eb669b469e033aaad9e - Sigstore transparency entry: 2617695158
- Sigstore integration time:
-
Permalink:
OvertureMaps/overture-core@20ad654267068b7e30a9f369ed14bd12e00126d2 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/OvertureMaps
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish_packages.yml@20ad654267068b7e30a9f369ed14bd12e00126d2 -
Trigger Event:
push
-
Statement type: