Machine Learning Orchestration
Project description
Overview
The dbnd-mlflow
plugin allows storing mlflow metrics to DBND tracker together with duplicating them to the mlflow store.
Install
pip install dbnd-mlflow
# or
pip install databand[mlflow]
Config
[core]
# Databand store url should be defined
databand_url=http://localhost:8080
[mlflow_tracking]
# Enable tracking to Databand store
databand_tracking=True
# Optionally, define a URI for mlflow store,
# mlflow.get_tracking_uri() is used by default
; duplicate_tracking_to=http://mlflow-store/
Run example
You might need to install examples at first pip install dbnd-examples
.
dbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow
# or set configs manually
dbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow --set-config mlflow_tracking.databand_tracking=True
Explanation
mlflow_example code
from dbnd import task
from mlflow import start_run, end_run
from mlflow import log_metric, log_param
@task
def mlflow_example():
start_run()
# params
log_param("param1", randint(0, 100))
log_param("param2", randint(0, 100))
# metrics
log_metric("foo1", random())
log_metric("foo2", random())
end_run()
Execution flow:
- Run
dbnd run mlflow_example --set-config mlflow_tracking.databand_tracking=True
- dbnd creates a new dbnd context
dbnd_on_pre_init_context
hook fromdbnd_mlflow
is triggered- a new uri is computed to be used by mlflow, e.g.:
dbnd://localhost:8080?duplicate_tracking_to=http%253A%252F%252Fmlflow-store%253A80%252F
- the new uri is set to be used with
mlflow.set_tracking_uri()
- a new uri is computed to be used by mlflow, e.g.:
mlflow_example
task starts:mlflow.start_run()
mlflow
readsentry_points
for each installed package and finds:- "dbnd = dbnd_mlflow.tracking_store:get_dbnd_store",
- "dbnd+s = dbnd_mlflow.tracking_store:get_dbnd_store",
- "databand = dbnd_mlflow.tracking_store:get_dbnd_store",
- "databand+s = dbnd_mlflow.tracking_store:get_dbnd_store",
mlflow
createsTrackingStoreClient
using the new uri- uri schema instructs to use
dbnd_mlflow.tracking_store:get_dbnd_store
get_dbnd_store
creates dbndTrackingAPIClient
get_dbnd_store
creates mlflow tracking store to duplicate tracking toget_dbnd_store
returnsDatabandStore
instance
log_param()
/log_metric()
- calls to
DatabandStore
- calls to
TrackingAPIClient
- calls to mlflow tracking store to duplicate tracking to
- calls to
- calls to
mlflow.end_run()
mlflow_example
endsdbnd_on_exit_context
hook fromdbnd_mlflow
is triggered- restore original mlflow tracking uri
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
Close
Hashes for dbnd_mlflow-0.82.9-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1af5eb17fd55afc85b5a62c7e0089c0f115a0614025d73add17d8519eadfe48a |
|
MD5 | e6d7b3781914477afabb5253b574e9d3 |
|
BLAKE2b-256 | 424a2f443b85f541c19b2eb5b935d3513d361c6c20b855f336099d3f6ae791a8 |