Skip to main content

No project description provided

Project description

ci/cd: publish snapshot to PyPI PyPI version license=mit Ideas and feedback=welcome

pynb-dag-runner

pynb-dag-runner is a Python library that can run ML/data pipelines on stateless compute infrastructure (that may be ephemeral or serverless).

This means that pynb-dag-runner does not need a tracking server (or database) to record task outcomes (like logged ML metrics, models, artifacts). Instead, pipeline outputs are emitted using the OpenTelemetry standard. Since structured logs can be directed to a file (as one option), this can be used to run pipelines on limited or no cloud infrastructure; after pipeline execution one only needs to preserve the structured logs.

Documentation and architecture

Demo

Roadmap and project planning

Install via PyPI

Latest release

Snapshot of latest commit to main branch


Any feedback/ideas welcome!

License

(c) Matias Dahl 2021-2022, MIT, see LICENSE.md.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

File details

Details for the file composable_logs_snapshot-0.0.10.dev1674908374-py3-none-any.whl.

File metadata

File hashes

Hashes for composable_logs_snapshot-0.0.10.dev1674908374-py3-none-any.whl
Algorithm Hash digest
SHA256 1c40b553279c35aacfef80a096f1a7194fda5736846a1b64352ba8e22fecad74
MD5 1ac57a639c6b86bd024c0f5d48c98322
BLAKE2b-256 8dd1e7f4ded33b6f6d90eb078bd92573c323d6ac70fd3fe54f0b9b229efea59d

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page