This release is a pre-release and may not be stable for production use.
mlpipeline
This is a simple framework to organize you machine learning workflow. It automates most of the basic functionalities such as logging, a framework for testing models and gluing together different steps at different stages. This project came about as a result of me abstracting the boilerplate code and automating different parts of the process.
The aim of this simple framework is to consolidate the different sub-problems (such as loading data, model configurations, training process, evaluation process, exporting trained models, etc.) when working/researching with machine learning models. This allows the user to define how the different sub-problems are to be solved using their choice of tools and mlpipeline would handle piecing them together.
Core operations
This framework chains the different operations (sub-problems) depending on the mode it is executed in. mlpipeline currently has 3 modes:
- TEST mode: When in TEST mode, it doesn't perform any logging or tracking. It creates a temporary empty directory for the experiment to store the artifacts of an experiment in. When developing and testing the different operations, this mode can be used.
- RUN mode: In this mode, logging and tracking is performed. In addition, for each experiment run (referred to as a experiment version in mlpipeline) a directory is created for artifacts to be stored.
- EXPORT mode: In this mode, the exporting related operations will be executed instead of the training/evaluation related operations.
In addition to providing different modes, the pipeline also supports logging and recording various details. Currently mlpipeline records all logs, metrics and artifacts using a basic log files as well using mlflow <https://github.com/databricks/mlflow>_.
The following information is recorded:
- The scripts that were executed/imported in relation to an experiment.
- The any output results
- The metrics and parameters
Documentation
The documentation is hosted at ReadTheDocs <https://mlpipeline.readthedocs.io/>_.
Installing
Can be installed directly using the Python Package Index using pip::
pip install mlpipeline
Usage
work in progress
1 maintainer
Release files for mlpipeline 2.0a4.post10
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| mlpipeline-2.0a4.post10.tar.gz | 24.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| mlpipeline-2.0a4.post10-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 53.6 kB
Release files / mlpipeline-2.0a4.post10.tar.gz
| Download URL | mlpipeline-2.0a4.post10.tar.gz |
|---|---|
| Size | 24.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a69688e159ddb0ffc0a7d424c7c5351a03b39cfd23db003bddcb30f35c78f014
|
|
BLAKE2b-256 checksum How to use checksums |
4e01b6f70f46c6f0fb97fe0109f727d5f0aff413df71a821ba16c3dfc50eb1b2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.18.4 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/3.7.3
|
Release files / mlpipeline-2.0a4.post10-py3-none-any.whl
| Download URL | mlpipeline-2.0a4.post10-py3-none-any.whl |
|---|---|
| Size | 29.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0ced61eefe85ee70d39a8d924223535a5619e496384a62808278d95a2dc322c5
|
|
BLAKE2b-256 checksum How to use checksums |
a50ab962811b490d727ff415898d2c98837ce8e1a1f90cc9ac5159c74d3d4450
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.18.4 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/3.7.3
|
