Package for interaction with MLFlow Registry
Project description
MLFlow model downloader
Allows to download models from MLFlow Model Registry using model's status (Staging or Production) or model's Version.
NOTES:
- supports MLflow's version <1.20.0
- tested only with Tensorflow/Keras models
- tested on S3 and SFTP storages
Installation
- Installation for production from cloned repository
cd mlflow_dl
pip install .
- Installation for development from cloned repository
cd mlflow_dl
pip install -e ".[dev]"
Environment Variables Configuration
In the most of the cases you need to configure a proper credentials to mlflow and related storage. Built in variables:
ENV Name | Description |
---|---|
MLFLOWDL_TARGET_FOLDER_LOCAL | The folder where downloaded results placed |
Also, you may need to configure variable for access to MLFlow and AWS S3 bucket. Some of them:
ENV Name | Description |
---|---|
AWS_ACCESS_KEY_ID | |
AWS_SECRET_ACCESS_KEY | |
AWS_DEFAULT_REGION | |
AWS_BUCKET_NAME | |
MLFLOW_TRACKING_USERNAME | Username for HTTP Base Auth for MLFlow (if used) |
MLFLOW_TRACKING_PASSWORD | Password for HTTP Base Auth for MLFlow (if used) |
MLFLOW_TRACKING_URI | URL to MLFlow UI (required) |
Usage from Command Line
- Get list of available commands:
python -m mlflow_dl
or
mlflow_dl
- Download models in "Production" stage:
mlflow_dl download-models-production-latest -m model_name_1,model_name_2
- Download models in "Staging" stage:
mlflow_dl download-models-staging-latest -m model_name_1,model_name_2
- Download model by name and version number:
mlflow_dl download-model-by-version -m model_name -v 1
- Download specific folder from the root of model's experiment:
mlflow_dl download-folder-by-model-version -m model_name -v 1 -f custom_folder
- Download specific folder for Tensorflow Serving compatible structure:
mlflow_dl download-folder-by-model-version -m model_name -v 1 -f custom_folder --no-subfolder
Usage from Code
from mlflow_dl import MlflowDl
mlflowdl = MlflowDl(target_folder="tmp_folder")
mlflowdl.download_model_by_version("model_name", "1")
Development
Testing
python -m unittest
Coverage
python -m coverage run -m unittest
python -m coverage html -i
Code Style
python -m flake8
Versioning
Commit your changes and run with a proper label (major | minor | patch):
bumpversion patch
git push origin
git push origin --tags
Build package
rm -rf ./dist
python -m build
Upload to custom PyPI
In case of private repo you need to make sure you configured credentials in ~/.pypirc
, than run:
twine upload --repository {repo_name} dist/*
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
mlflow_dl-0.2.0.tar.gz
(6.3 kB
view hashes)
Built Distribution
Close
Hashes for mlflow_dl-0.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 98f0f6b09b80e4246b44e0d4be5536c1bcb64758397547523ed0dfc8d642795c |
|
MD5 | dafc953eb96944d28ccc77f876eb1e0a |
|
BLAKE2b-256 | 263f8d8d375e81bd551e608cce8cdba42aca409dccfe7033f8b57c9e671b558c |