Machine Learning Orchestration
Project description
Overview
The dbnd-mlflow
plugin allows storing mlflow metrics to DBND tracker together with duplicating them to the mlflow store.
Install
pip install dbnd-mlflow
# or
pip install databand[mlflow]
Config
[core]
# Databand store url should be defined
databand_url=http://localhost:8080
[mlflow_tracking]
# Enable tracking to Databand store
databand_tracking=True
# Optionally, define a URI for mlflow store,
# mlflow.get_tracking_uri() is used by default
; duplicate_tracking_to=http://mlflow-store/
Run example
You might need to install examples at first pip install dbnd-examples
.
dbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow
# or set configs manually
dbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow --set-config mlflow_tracking.databand_tracking=True
Explanation
mlflow_example code
from dbnd import task
from mlflow import start_run, end_run
from mlflow import log_metric, log_param
@task
def mlflow_example():
start_run()
# params
log_param("param1", randint(0, 100))
log_param("param2", randint(0, 100))
# metrics
log_metric("foo1", random())
log_metric("foo2", random())
end_run()
Execution flow:
- Run
dbnd run mlflow_example --set-config mlflow_tracking.databand_tracking=True
- dbnd creates a new dbnd context
dbnd_on_pre_init_context
hook fromdbnd_mlflow
is triggered- a new uri is computed to be used by mlflow, e.g.:
dbnd://localhost:8080?duplicate_tracking_to=http%253A%252F%252Fmlflow-store%253A80%252F
- the new uri is set to be used with
mlflow.set_tracking_uri()
- a new uri is computed to be used by mlflow, e.g.:
mlflow_example
task starts:mlflow.start_run()
mlflow
readsentry_points
for each installed package and finds:- "dbnd = dbnd_mlflow.tracking_store:get_dbnd_store",
- "dbnd+s = dbnd_mlflow.tracking_store:get_dbnd_store",
- "databand = dbnd_mlflow.tracking_store:get_dbnd_store",
- "databand+s = dbnd_mlflow.tracking_store:get_dbnd_store",
mlflow
createsTrackingStoreClient
using the new uri- uri schema instructs to use
dbnd_mlflow.tracking_store:get_dbnd_store
get_dbnd_store
creates dbndTrackingAPIClient
get_dbnd_store
creates mlflow tracking store to duplicate tracking toget_dbnd_store
returnsDatabandStore
instance
log_param()
/log_metric()
- calls to
DatabandStore
- calls to
TrackingAPIClient
- calls to mlflow tracking store to duplicate tracking to
- calls to
- calls to
mlflow.end_run()
mlflow_example
endsdbnd_on_exit_context
hook fromdbnd_mlflow
is triggered- restore original mlflow tracking uri
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
dbnd_mlflow-1.0.25.2.tar.gz
(12.3 kB
view hashes)
Built Distribution
Close
Hashes for dbnd_mlflow-1.0.25.2-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | aa51a14aa2c3cd4cdac7084684dd36430683a6baccbb1e45532b9660dcd88c46 |
|
MD5 | 6d0e03a0ec13b5d90dbdfd07afb98907 |
|
BLAKE2b-256 | da3fcbf94bc3d955ea0e51cfaeda40fa81903936803cbee2aca6b631700d0752 |