Package for interaction with MLFlow Registry
Project description
MLFlow model downloader
Allows to download models from MLFlow Model Registry using model's status (Staging or Production) or model's Version.
NOTES:
- supports MLflow's version <1.20.0
- tested only with Tensorflow/Keras models
- tested on S3 and SFTP storages
Installation
- Installation for production from cloned repository
cd mlflow_dl
pip install .
- Installation for development from cloned repository
cd mlflow_dl
pip install -e ".[dev]"
Environment Variables Configuration
In the most of the cases you need to configure a proper credentials to mlflow and related storage. Built in variables:
ENV Name | Description |
---|---|
MLFLOWDL_TARGET_FOLDER_LOCAL | The folder where downloaded results placed |
Also, you may need to configure variable for access to MLFlow and AWS S3 bucket. Some of them:
ENV Name | Description |
---|---|
AWS_ACCESS_KEY_ID | |
AWS_SECRET_ACCESS_KEY | |
AWS_DEFAULT_REGION | |
AWS_BUCKET_NAME | |
MLFLOW_TRACKING_USERNAME | Username for HTTP Base Auth for MLFlow (if used) |
MLFLOW_TRACKING_PASSWORD | Password for HTTP Base Auth for MLFlow (if used) |
MLFLOW_TRACKING_URI | URL to MLFlow UI (required) |
Usage from Command Line
- Get list of available commands:
python -m mlflow_dl
or
mlflow_dl
- Download models in "Production" stage:
mlflow_dl download-models-production-latest -m model_name_1,model_name_2
- Download models in "Staging" stage:
mlflow_dl download-models-staging-latest -m model_name_1,model_name_2
- Download model by name and version number:
mlflow_dl download-model-by-version -m model_name -v 1
- Download specific folder from the root of model's experiment:
mlflow_dl download-folder-by-model-version -m model_name -v 1 -f custom_folder
- Download specific folder for Tensorflow Serving compatible structure:
mlflow_dl download-folder-by-model-version -m model_name -v 1 -f custom_folder --no-subfolder
Usage from Code
from mlflow_dl import MlflowDl
mlflowdl = MlflowDl(target_folder="tmp_folder")
mlflowdl.download_model_by_version("model_name", "1")
Development
Testing
python -m unittest
Coverage
python -m coverage run -m unittest
python -m coverage html -i
Code Style
python -m flake8
Versioning
Commit your changes and run with a proper label (major | minor | patch):
bumpversion patch
git push origin
git push origin --tags
Build package
rm -rf ./dist
python -m build
Upload to custom PyPI
In case of private repo you need to make sure you configured credentials in ~/.pypirc
, than run:
twine upload --repository {repo_name} dist/*
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
mlflow_dl-0.2.1.tar.gz
(6.2 kB
view hashes)
Built Distribution
Close
Hashes for mlflow_dl-0.2.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 48a180246a353c90c8337bcf3f9c571ba9ea49a2578cce4b6fca5144fef81cee |
|
MD5 | 36242c4897dc4abaae6de90f5097b730 |
|
BLAKE2b-256 | 33e2af83c904298c888527344ca8e12935321b1e2d8cf92f55607070d893be3d |