Skip to main content

No project description provided

Project description

ci/cd: publish snapshot to PyPI PyPI version license=mit Ideas and feedback=welcome

pynb-dag-runner

pynb-dag-runner is a Python library that can run ML/data pipelines on stateless compute infrastructure (that may be ephemeral or serverless).

This means that pynb-dag-runner does not need a tracking server (or database) to record task outcomes (like logged ML metrics, models, artifacts). Instead, pipeline outputs are emitted using the OpenTelemetry standard. Since structured logs can be directed to a file (as one option), this can be used to run pipelines on limited or no cloud infrastructure; after pipeline execution one only needs to preserve the structured logs.

Documentation and architecture

Demo

Roadmap and project planning

Install via PyPI

Latest release

Snapshot of latest commit to main branch


Any feedback/ideas welcome!

License

(c) Matias Dahl 2021-2022, MIT, see LICENSE.md.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

File details

Details for the file composable_logs_snapshot-0.0.10.dev1673772257-py3-none-any.whl.

File metadata

File hashes

Hashes for composable_logs_snapshot-0.0.10.dev1673772257-py3-none-any.whl
Algorithm Hash digest
SHA256 4ff1ac1f227eb27358b39b754f5d28846b8e0bad232216ffde5aba4f5bbb4474
MD5 91c21b05378c3ca59ea3c9ab31252a0d
BLAKE2b-256 3104abd490e10b92d9d20403ef45861094495a7265b6f4f1adc64a6b350c3042

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page