Skip to main content

No project description provided

Project description

ci/cd: publish snapshot to PyPI PyPI version license=mit Ideas and feedback=welcome

pynb-dag-runner

pynb-dag-runner is a Python library that can run ML/data pipelines on stateless compute infrastructure (that may be ephemeral or serverless).

This means that pynb-dag-runner does not need a tracking server (or database) to record task outcomes (like logged ML metrics, models, artifacts). Instead, pipeline outputs are emitted using the OpenTelemetry standard. Since structured logs can be directed to a file (as one option), this can be used to run pipelines on limited or no cloud infrastructure; after pipeline execution one only needs to preserve the structured logs.

Documentation and architecture

Demo

Roadmap and project planning

Install via PyPI

Latest release

Snapshot of latest commit to main branch


Any feedback/ideas welcome!

License

(c) Matias Dahl 2021-2022, MIT, see LICENSE.md.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

File details

Details for the file composable_logs_snapshot-0.0.10.dev1673000951-py3-none-any.whl.

File metadata

File hashes

Hashes for composable_logs_snapshot-0.0.10.dev1673000951-py3-none-any.whl
Algorithm Hash digest
SHA256 431fd7e12c5e6ec3d639dc868552b78ba41ee900240bdc133bc83a6b4e094dd9
MD5 ac830607bf162eeb807621f6223e0a96
BLAKE2b-256 4f62877bb4b93c48a8e17fa1e32e6b764e5302356773f6750d27b3d23919746b

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page