Skip to main content

No project description provided

Project description

ci/cd: publish snapshot to PyPI PyPI version license=mit Ideas and feedback=welcome

pynb-dag-runner

pynb-dag-runner is a Python library that can run ML/data pipelines on stateless compute infrastructure (that may be ephemeral or serverless).

This means that pynb-dag-runner does not need a tracking server (or database) to record task outcomes (like logged ML metrics, models, artifacts). Instead, pipeline outputs are emitted using the OpenTelemetry standard. Since structured logs can be directed to a file (as one option), this can be used to run pipelines on limited or no cloud infrastructure; after pipeline execution one only needs to preserve the structured logs.

Documentation and architecture

Demo

Roadmap and project planning

Install via PyPI

Latest release

Snapshot of latest commit to main branch


Any feedback/ideas welcome!

License

(c) Matias Dahl 2021-2022, MIT, see LICENSE.md.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

File details

Details for the file composable_logs_snapshot-0.0.10.dev1673795260-py3-none-any.whl.

File metadata

File hashes

Hashes for composable_logs_snapshot-0.0.10.dev1673795260-py3-none-any.whl
Algorithm Hash digest
SHA256 6e683c4d203509a8442533347bbde3ad285d7f418daa0eddd51d8c42e4d15b8a
MD5 7f929045b009655a248a3c6e4af83a1c
BLAKE2b-256 3f30b15bd4561ea290b5ef74958e8457e1420316fd11bd3b4f5c2d6a685e377e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page