Skip to main content

No project description provided

Project description

ci/cd: publish snapshot to PyPI PyPI version license=mit Ideas and feedback=welcome

pynb-dag-runner

pynb-dag-runner is a Python library that can run ML/data pipelines on stateless compute infrastructure (that may be ephemeral or serverless).

This means that pynb-dag-runner does not need a tracking server (or database) to record task outcomes (like logged ML metrics, models, artifacts). Instead, pipeline outputs are emitted using the OpenTelemetry standard. Since structured logs can be directed to a file (as one option), this can be used to run pipelines on limited or no cloud infrastructure; after pipeline execution one only needs to preserve the structured logs.

Documentation and architecture

Demo

Roadmap and project planning

Install via PyPI

Latest release

Snapshot of latest commit to main branch


Any feedback/ideas welcome!

License

(c) Matias Dahl 2021-2022, MIT, see LICENSE.md.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

File details

Details for the file composable_logs_snapshot-0.0.10.dev1674409880-py3-none-any.whl.

File metadata

File hashes

Hashes for composable_logs_snapshot-0.0.10.dev1674409880-py3-none-any.whl
Algorithm Hash digest
SHA256 f6e713fdc46006e0b0a6494abf8298c8796970fa5754fb1000736df3cee9306d
MD5 8ea64031e053368ecbaab26947fb3ef9
BLAKE2b-256 76e5efc1130058475b5f31bd6fceacb31d451505b87e8a8105e87d96e2d6e291

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page