Simple and composable metric tracking and logging for ML
Project description
Demetric
Dead simple standard for metric logging
Logging
import demetric as dm
metrics = dm.Metrics.new('metrics/gpt2-3')
metrics.log('loss', value=loss, step=step)
metrics.log('accuracy', value=acc, step=step)
# ...
Creates:
runs/
gpt2-3/
loss.csv
accuracy.csv
Reading
# single metrics
metrics = dm.Metrics('metrics/run1.0')
metrics.read('loss') # pd.DataFrame
# all metrics
metrics.read() # pd.DataFrame with a column per metric
# comparing runs
runs = { run: dm.Metrics(run) for run in ['metrics/version1', 'metrics/version2'] }
df = dm.compare(runs, 'loss') # pd.DataFrame with columns ("loss_run1.0", "loss_run1.1", ...)
# concatenating runs (i.e. they're the same experiment but trained by steps or something)
runs = [dm.Metrics(run) for run in ['metrics/part1', 'metrics/part2']]
df = dm.concat(runs, 'loss') # pd.DataFrame with cumulative step indices
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
demetric-0.1.11.tar.gz
(3.2 kB
view hashes)
Built Distribution
Close
Hashes for demetric-0.1.11-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 197dbbc9cdf7c5f3498b8d004bc8e75aa4bb236b10a7ec5e0ef79288c5f0cd21 |
|
MD5 | 6849d55e1afa144806f8ddfdff03b590 |
|
BLAKE2b-256 | 2fabc71f98a598ad081a69aeaee6f336087a5d7ae9efb82e62ce0fd6ad1484a6 |