Reduce multiple tensorboard runs to new summary or CSV files
Project description
Compute reduced statistics (mean
, std
, min
, max
, median
or any other numpy
operation) of multiple TensorBoard runs matching a directory glob pattern. This can be used after training multiple identical models to reduce the noise in their loss/accuracy/error curves e.g. when trying to establish a statistically significant improvement in training performance.
Requires PyTorch and TensorBoard. No TensorFlow installation required.
Installation
pip install tensorboard_reducer
Usage
The tb-reducer
CLI has the following flags:
-i/--indirs-glob
(required): Glob pattern of the run directories to reduce.-o/--outdir
(required): Name of the directory to save the new reduced run data.-r/--reduce-ops
(required): Comma-separated names of numpy reduction ops (mean
,std
,min
,max
, ...). Default ismean
. Each reduction is written to a separateoutdir
suffixed by its op name, e.g. ifoutdir='my-new-run
, the mean reduction will be written tomy-new-run-mean
. Only exception isstd
which will create twooutdir
s namedmy-new-run-mean+std
andmy-new-run-mean-std
.-f/--format
: Output format of reduced TensorBoard runs. One oftb-events
for regular TensorBoard event files orcsv
. Ifcsv
,-o/--outdir
must have.csv
extension and all reduction ops will be written to a single CSV file rather than separate directories for each reduce op.-w/--overwrite
(optional): Whether to overwrite existing reduction directories.
Example:
tb-reducer -i 'glob_pattern/of_dirs_to_reduce*' -o basename_of_output_dir -r mean,std,min,max
Testing
This project uses pytest
. To run the entire test suite:
python -m pytest
To run individual or groups of test files, pass pytest
a path or glob pattern, respectively:
python -m pytest tests/test_cumulative.py
python -m pytest **/test_*_metrics.py
To run a single test, pass its name to the -k
flag:
python -m pytest -k test_precision_recall_curve
Consult the pytest
docs for more details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tensorboard-reducer-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0d7c24b26a16fa1785c58f1398a7d1cb5b796585fea49f714d7026528d93d2af |
|
MD5 | 7db7f72f5e1f6ee3910b87c9805d6b60 |
|
BLAKE2b-256 | 218a99440ed4248ba2ae0b77b0a18444aba5911707645859a9ca118127831cb8 |
Hashes for tensorboard_reducer-0.1.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c527aeff8ffcf5797ae990841429baba37531d0c4f5a4333f91872e0a156aff1 |
|
MD5 | 89eadacdcee97c6c20722c86a08ebe7b |
|
BLAKE2b-256 | 4e7f24aeaf0b66c302152cca8d58cc5f1597e31a9e6af26e3fad7088035f4f47 |