Machine Learning Orchestration
Project description
Overview
The dbnd-mlflow
plugin allows storing mlflow metrics to DBND tracker together with duplicating them to the mlflow store.
Install
pip install dbnd-mlflow
# or
pip install databand[mlflow]
Config
[core]
# Databand store url should be defined
databand_url=http://localhost:8080
[mlflow_tracking]
# Enable tracking to Databand store
databand_tracking=True
# Optionally, define a URI for mlflow store,
# mlflow.get_tracking_uri() is used by default
; duplicate_tracking_to=http://mlflow-store/
Run example
You might need to install examples at first pip install dbnd-examples
.
dbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow
# or set configs manually
dbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow --set-config mlflow_tracking.databand_tracking=True
Explanation
mlflow_example code
from dbnd import task
from mlflow import start_run, end_run
from mlflow import log_metric, log_param
@task
def mlflow_example():
start_run()
# params
log_param("param1", randint(0, 100))
log_param("param2", randint(0, 100))
# metrics
log_metric("foo1", random())
log_metric("foo2", random())
end_run()
Execution flow:
- Run
dbnd run mlflow_example --set-config mlflow_tracking.databand_tracking=True
- dbnd creates a new dbnd context
dbnd_on_pre_init_context
hook fromdbnd_mlflow
is triggered- a new uri is computed to be used by mlflow, e.g.:
dbnd://localhost:8080?duplicate_tracking_to=http%253A%252F%252Fmlflow-store%253A80%252F
- the new uri is set to be used with
mlflow.set_tracking_uri()
- a new uri is computed to be used by mlflow, e.g.:
mlflow_example
task starts:mlflow.start_run()
mlflow
readsentry_points
for each installed package and finds:- "dbnd = dbnd_mlflow.tracking_store:get_dbnd_store",
- "dbnd+s = dbnd_mlflow.tracking_store:get_dbnd_store",
- "databand = dbnd_mlflow.tracking_store:get_dbnd_store",
- "databand+s = dbnd_mlflow.tracking_store:get_dbnd_store",
mlflow
createsTrackingStoreClient
using the new uri- uri schema instructs to use
dbnd_mlflow.tracking_store:get_dbnd_store
get_dbnd_store
creates dbndTrackingAPIClient
get_dbnd_store
creates mlflow tracking store to duplicate tracking toget_dbnd_store
returnsDatabandStore
instance
log_param()
/log_metric()
- calls to
DatabandStore
- calls to
TrackingAPIClient
- calls to mlflow tracking store to duplicate tracking to
- calls to
- calls to
mlflow.end_run()
mlflow_example
endsdbnd_on_exit_context
hook fromdbnd_mlflow
is triggered- restore original mlflow tracking uri
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
dbnd-mlflow-0.79.7.tar.gz
(11.3 kB
view hashes)
Built Distribution
Close
Hashes for dbnd_mlflow-0.79.7-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 81ca04740f2dc17282200d93cd4a9446cc0bf690dcd945556008630edeb85c9a |
|
MD5 | 154f8f77e84ba797a820e5fcad0865da |
|
BLAKE2b-256 | edb391c572f0b41200641ed3375cd1a6ebc840ce1e45722fd2d8cfabc1df975e |