Simple and composable metric tracking and logging for ML
Project description
Demetric
Dead simple standard for metric logging
Logging
import demetric as dm
run = dm.Run.new('runs/gpt2-3')
run.log('loss', value=loss, step=step)
run.log('accuracy', value=acc, step=step)
# ...
Creates:
runs/
gpt2-3/
loss.csv
accuracy.csv
Statistics
# single run
run = dm.Run('runs/run1.0')
run.read('loss') # pd.Series
# comparing runs
runs = dm.runs('runs/run1.*')
df = dm.compare(runs, 'loss') # pd.DataFrame with columns ("run1.0", "run1.1", ...)
# concatenating runs (i.e. they're the same experiment but trained by steps or something)
df = dm.concat(runs, 'loss') # pd.Series with cumulative step indices
# whatever you want to do with the pd.Series's
ss = dm.readall(runs, 'loss') # list[pd.Series] with `run.id` as each name
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
demetric-0.1.7.tar.gz
(3.1 kB
view hashes)