Machine Learning Orchestration
Project description
Overview
The dbnd-mlflow
plugin allows storing mlflow metrics to DBND tracker together with duplicating them to the mlflow store.
Install
pip install dbnd-mlflow
# or
pip install databand[mlflow]
Config
[core]
# Databand store url should be defined
databand_url=http://localhost:8081
[mlflow_tracking]
# Enable tracking to Databand store
databand_tracking=True
# Optionally, define a URI for mlflow store,
# mlflow.get_tracking_uri() is used by default
; duplicate_tracking_to=http://mlflow-store/
Run example
You might need to install examples at first pip install dbnd-examples
.
dbnd run dbnd_examples.mlflow.run_mlflow_in_dbnd_task.mlflow_tracking_in_task_example
# or set configs manually
dbnd run dbnd_examples.mlflow.run_mlflow_in_dbnd_task.mlflow_tracking_in_task_example --set-config mlflow_tracking.databand_tracking=True
Explanation
mlflow_example code
from dbnd import task
from mlflow import start_run, end_run
from mlflow import log_metric, log_param
@task
def mlflow_example():
start_run()
# params
log_param("param1", randint(0, 100))
log_param("param2", randint(0, 100))
# metrics
log_metric("foo1", random())
log_metric("foo2", random())
end_run()
Execution flow:
- Run
dbnd run mlflow_example --set-config mlflow_tracking.databand_tracking=True
- dbnd creates a new dbnd context
dbnd_on_pre_init_context
hook fromdbnd_mlflow
is triggered- a new uri is computed to be used by mlflow, e.g.:
dbnd://localhost:8081?duplicate_tracking_to=http%253A%252F%252Fmlflow-store%253A80%252F
- the new uri is set to be used with
mlflow.set_tracking_uri()
- a new uri is computed to be used by mlflow, e.g.:
mlflow_example
task starts:mlflow.start_run()
mlflow
readsentry_points
for each installed package and finds:- "dbnd = dbnd_mlflow.tracking_store:get_dbnd_store",
- "dbnd+s = dbnd_mlflow.tracking_store:get_dbnd_store",
- "databand = dbnd_mlflow.tracking_store:get_dbnd_store",
- "databand+s = dbnd_mlflow.tracking_store:get_dbnd_store",
mlflow
createsTrackingStoreClient
using the new uri- uri schema instructs to use
dbnd_mlflow.tracking_store:get_dbnd_store
get_dbnd_store
creates dbndTrackingAPIClient
get_dbnd_store
creates mlflow tracking store to duplicate tracking toget_dbnd_store
returnsDatabandStore
instance
log_param()
/log_metric()
- calls to
DatabandStore
- calls to
TrackingAPIClient
- calls to mlflow tracking store to duplicate tracking to
- calls to
- calls to
mlflow.end_run()
mlflow_example
endsdbnd_on_exit_context
hook fromdbnd_mlflow
is triggered- restore original mlflow tracking uri
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
dbnd-mlflow-0.24.26.tar.gz
(10.5 kB
view hashes)
Built Distribution
Close
Hashes for dbnd_mlflow-0.24.26-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c1fb72c855d1eaf78a0fba051c7ac50bf49a444d4e11b0a3f3d811a1e811df90 |
|
MD5 | fdfe1100c0b5cfa50f31269bbcc9b20f |
|
BLAKE2b-256 | f89fc1da5ad68d3154ffdc77b069350977ccf7ca5caa2eadef87fe665361c317 |