Reduce multiple TensorBoard runs to new event (or CSV) files
Project description
Compute reduced statistics (mean
, std
, min
, max
, median
or any other numpy
operation) of multiple TensorBoard runs matching a directory glob pattern. This can be used after training multiple identical models to reduce the noise in their loss/accuracy/error curves e.g. when trying to establish a statistically significant improvement in training performance.
Requires PyTorch and TensorBoard. No TensorFlow installation required.
Installation
pip install tensorboard-reducer
Usage
Example:
tb-reducer -i 'glob_pattern/of_dirs_to_reduce*' -o basename_of_output_dir -r mean,std,min,max
tb-reducer
has the following flags:
-i/--indirs-glob
(required): Glob pattern of the run directories to reduce.-o/--outdir
(required): Name of the directory to save the new reduced run data.-r/--reduce-ops
(required): Comma-separated names of numpy reduction ops (mean
,std
,min
,max
, ...). Default ismean
. Each reduction is written to a separateoutdir
suffixed by its op name, e.g. ifoutdir='my-new-run
, the mean reduction will be written tomy-new-run-mean
. Only exception isstd
which will create twooutdir
s namedmy-new-run-mean+std
andmy-new-run-mean-std
.-f/--format
: Output format of reduced TensorBoard runs. One oftb-events
for regular TensorBoard event files orcsv
. Ifcsv
,-o/--outdir
must have.csv
extension and all reduction ops will be written to a single CSV file rather than separate directories for each reduce op.-w/--overwrite
(optional): Whether to overwrite existing reduction directories.
Testing
This project uses pytest
. To run the entire test suite:
python -m pytest
To run individual or groups of test files, pass pytest
a path or glob pattern, respectively:
python -m pytest tests/test_cumulative.py
python -m pytest **/test_*_metrics.py
To run a single test, pass its name to the -k
flag:
python -m pytest -k test_precision_recall_curve
Consult the pytest
docs for more details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tensorboard-reducer-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8e17868d2af068484295f3e27841f8217698dd52c6a4dc1591ea075a5c4a3638 |
|
MD5 | 028135117c91bcfc7c3ebaade0cb19c7 |
|
BLAKE2b-256 | 43fb68daf668e506c1b8fb6a376bdd1ccbfee2f7a60c246e399e528e37d02472 |
Hashes for tensorboard_reducer-0.1.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 27fda5378b8048567db0ededc4b8bc763ba4e3a91448c729868c0449764db313 |
|
MD5 | d537cd0e2baf537795e869239f800d9e |
|
BLAKE2b-256 | c09a8cc5be952a8a4e549c615e75967e70dac4fd8c8246b5d10e0f4f75727fb9 |