Machine Learning Orchestration
Project description
Overview
The dbnd-mlflow
plugin allows storing mlflow metrics to DBND tracker together with duplicating them to the mlflow store.
Install
pip install dbnd-mlflow
# or
pip install databand[mlflow]
Config
[core]
# Databand store url should be defined
databand_url=http://localhost:8080
[mlflow_tracking]
# Enable tracking to Databand store
databand_tracking=True
# Optionally, define a URI for mlflow store,
# mlflow.get_tracking_uri() is used by default
; duplicate_tracking_to=http://mlflow-store/
Run example
You might need to install examples at first pip install dbnd-examples
.
dbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow
# or set configs manually
dbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow --set-config mlflow_tracking.databand_tracking=True
Explanation
mlflow_example code
from dbnd import task
from mlflow import start_run, end_run
from mlflow import log_metric, log_param
@task
def mlflow_example():
start_run()
# params
log_param("param1", randint(0, 100))
log_param("param2", randint(0, 100))
# metrics
log_metric("foo1", random())
log_metric("foo2", random())
end_run()
Execution flow:
- Run
dbnd run mlflow_example --set-config mlflow_tracking.databand_tracking=True
- dbnd creates a new dbnd context
dbnd_on_pre_init_context
hook fromdbnd_mlflow
is triggered- a new uri is computed to be used by mlflow, e.g.:
dbnd://localhost:8080?duplicate_tracking_to=http%253A%252F%252Fmlflow-store%253A80%252F
- the new uri is set to be used with
mlflow.set_tracking_uri()
- a new uri is computed to be used by mlflow, e.g.:
mlflow_example
task starts:mlflow.start_run()
mlflow
readsentry_points
for each installed package and finds:- "dbnd = dbnd_mlflow.tracking_store:get_dbnd_store",
- "dbnd+s = dbnd_mlflow.tracking_store:get_dbnd_store",
- "databand = dbnd_mlflow.tracking_store:get_dbnd_store",
- "databand+s = dbnd_mlflow.tracking_store:get_dbnd_store",
mlflow
createsTrackingStoreClient
using the new uri- uri schema instructs to use
dbnd_mlflow.tracking_store:get_dbnd_store
get_dbnd_store
creates dbndTrackingAPIClient
get_dbnd_store
creates mlflow tracking store to duplicate tracking toget_dbnd_store
returnsDatabandStore
instance
log_param()
/log_metric()
- calls to
DatabandStore
- calls to
TrackingAPIClient
- calls to mlflow tracking store to duplicate tracking to
- calls to
- calls to
mlflow.end_run()
mlflow_example
endsdbnd_on_exit_context
hook fromdbnd_mlflow
is triggered- restore original mlflow tracking uri
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
dbnd-mlflow-0.58.6.tar.gz
(10.6 kB
view hashes)
Built Distribution
Close
Hashes for dbnd_mlflow-0.58.6-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e80f46414b9aaf0f9ed504645334cd11dcb4e7d9b71c1e0061ca4e02a491cfb4 |
|
MD5 | 0711a232627db4905c66e3ac0f14fbce |
|
BLAKE2b-256 | 8fca071f9fb62e7de551c4966116e927a725b8d91fa3ac1383f143706649486b |