Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

mlpipeline

This is a simple frawork to organize you machine learning workflow. It automates most of the basic functionalities such as logging, a framework for testing models and gluing together different steps at different stages. This project came about as a result of me abstracting the boilerplate code and automating different parts of the process.

The aim of this simple framework is to consolidate the different sub-problems (such as loading data, model configurations, training process, evalutaion process, exporting trained models, etc.) when working/researching with machine learning models. This allows the user to define how the different sub-problems are to be solved using their choice of tools and mlpipeline would handle piecing them together.

Core operations

This framework chains the different operations (sub-problems) depending on the mode it is executed in. mlpipeline currently has 3 modes:

  • TEST mode: When in TEST mode, it doesn't perform any logging or tracking. It creates a temporory empty directory for the experiment to store the artifacts of an experiment in. When developing and testing the different operations, this mode can be used.
  • RUN mode: In this mode, logging and tracking is performed. In addition, for each experiment run (refered to as a experiment version in mlpipeline) a directory is created for artifacts to be stored.
  • EXPORT mode: In this mode, the exporting related operations will be executed instead of the training/evaluation related operations.

In addition to providing different modes, the pipeline also supports logging and recording various details. Currently mlpipeline records all logs, metrics and artifacts using a bacis log files as well using mlflow <https://github.com/databricks/mlflow>_.

The following information is recorded:

  • The scripts that were executed/impoerted in relation to an experiment.
  • The any output results
  • The metrics and parameters

Documentation

The documentation is hosted at ReadTheDocs <https://mlpipeline.readthedocs.io/>_.

Installing

Can be installed directly using the Python Package Index using pip: pip install mlpipeline

Usage

work in progress

Metadata

Release files for mlpipeline 2.0a3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for mlpipeline 2.0a3
File Size Uploaded Provenance Actions
mlpipeline-2.0a3.tar.gz 20.2 kB Without provenance Details for mlpipeline-2.0a3.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for mlpipeline 2.0a3
File Interpreter ABI Platform Actions
mlpipeline-2.0a3-py3-none-any.whl Python 3 none any Details for mlpipeline-2.0a3-py3-none-any.whl

Total release size: 44.5 kB

Release files / mlpipeline-2.0a3.tar.gz

Download URL mlpipeline-2.0a3.tar.gz
Size 20.2 kB
Tags Source
SHA-256 checksum
How to use checksums
e89275b895c507be254b6536e358bd4d759df1b633efd6b5e8e0f383bb581c48
BLAKE2b-256 checksum
How to use checksums
21c622307eaa5a471bf83c569da91cd67df2eb13ea590628383f66b0ab277d4a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.18.4 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/3.7.3

Release files / mlpipeline-2.0a3-py3-none-any.whl

Download URL mlpipeline-2.0a3-py3-none-any.whl
Size 24.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
6095af5ddeb96c3a48959751ff118c1e99952db2896167cab9bbf952cc8d0283
BLAKE2b-256 checksum
How to use checksums
ebaf826649ad62e11921c1f2bdb94c2c7fa38f5f91083dd8cccbfd4d3be41f5e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.18.4 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/3.7.3

Security for mlpipeline 2.0a3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page