Package for interaction with MLFlow Registry
Project description
MLFlow model downloader
Allows to download models from MLFlow Model Registry using model's status (Staging or Production) or model's Version.
NOTES:
- supports MLflow's version <1.20.0
- tested only with Tensorflow/Keras models
- tested on S3 and SFTP storages
Installation
Install from PyPi
pip install mlflow-dl
Install from GitHub
git clone git@github.com:dem-artem/mlflow_dl.git
cd mlflow_dl
- for production:
pip install .
- for development:
cd mlflow_dl
pip install -e ".[dev]"
Environment Variables Configuration
In the most of the cases you need to configure a proper credentials to mlflow and related storage. Built in variables:
ENV Name | Description |
---|---|
MLFLOWDL_TARGET_FOLDER_LOCAL | The folder where downloaded results placed |
Also, you may need to configure variables for access to MLFlow and AWS S3 bucket. Some of them:
ENV Name | Description |
---|---|
AWS_ACCESS_KEY_ID | |
AWS_SECRET_ACCESS_KEY | |
AWS_DEFAULT_REGION | |
AWS_BUCKET_NAME | |
MLFLOW_TRACKING_USERNAME | Username for HTTP Base Auth for MLFlow (if used) |
MLFLOW_TRACKING_PASSWORD | Password for HTTP Base Auth for MLFlow (if used) |
MLFLOW_TRACKING_URI | URL to MLFlow UI (required) |
Usage from Command Line
- Get list of available commands:
python -m mlflow_dl
or
mlflow_dl
- Download models in "Production" stage:
mlflow_dl download-production -m model_name_1,model_name_2
- Download models in "Staging" stage:
mlflow_dl download-staging -m model_name_1,model_name_2
- Download model by name and version number:
mlflow_dl download-model-by-version -m model_name -v 1
- Download specific folder from the root of model's experiment:
mlflow_dl download-folder-by-model-version -m model_name -v 1 -f custom_folder
- Download specific folder for Tensorflow Serving compatible structure:
mlflow_dl download-folder-by-model-version -m model_name -v 1 -f custom_folder --no-subfolder
Usage from Code
from mlflow_dl import MlflowDl
mlflowdl = MlflowDl(target_folder="tmp_folder")
mlflowdl.download_model_by_version("model_name", "1")
Development
Testing
python -m unittest
Coverage
python -m coverage run -m unittest
python -m coverage html -i
Code Style
python -m flake8
Versioning
Commit your changes and run with a proper label (major | minor | patch):
bumpversion patch
git push origin
git push origin --tags
Build package
rm -rf ./dist
python -m build
Upload to custom PyPI
In case of private repo you need to make sure you configured credentials in ~/.pypirc
, than run:
twine upload --repository {repo_name} dist/*
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
mlflow_dl-0.2.2.tar.gz
(6.2 kB
view hashes)
Built Distribution
Close
Hashes for mlflow_dl-0.2.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c059579d3bd28e9e7abda0b3dd8038d65ed50124380143dd0b2f9d43cf40c647 |
|
MD5 | 681e4290e1468ecba78baf84269a8e80 |
|
BLAKE2b-256 | f4bd583f8dc33359fc10103416bd5db7fff7a871053c25ed8590b4d869bae421 |