Reduce multiple TensorBoard runs to new event (or CSV) files
Project description
Compute reduced statistics (mean
, std
, min
, max
, median
or any other numpy
operation) of multiple TensorBoard runs matching a directory glob pattern. This can be used after training multiple identical models to reduce the noise in their loss/accuracy/error curves e.g. when trying to establish a statistically significant improvement in training performance.
Requires PyTorch and TensorBoard. No TensorFlow installation required.
Installation
pip install tensorboard-reducer
Usage
Example:
tb-reducer -i 'glob_pattern/of_dirs_to_reduce*' -o basename_of_output_dir -r mean,std,min,max
tb-reducer
has the following flags:
-i/--indirs-glob
(required): Glob pattern of the run directories to reduce.-o/--outdir
(required): Name of the directory to save the new reduced run data.-r/--reduce-ops
(required): Comma-separated names of numpy reduction ops (mean
,std
,min
,max
, ...). Default ismean
. Each reduction is written to a separateoutdir
suffixed by its op name, e.g. ifoutdir='my-new-run
, the mean reduction will be written tomy-new-run-mean
. Only exception isstd
which will create twooutdir
s namedmy-new-run-mean+std
andmy-new-run-mean-std
.-f/--format
: Output format of reduced TensorBoard runs. One oftb-events
for regular TensorBoard event files orcsv
. Ifcsv
,-o/--outdir
must have.csv
extension and all reduction ops will be written to a single CSV file rather than separate directories for each reduce op.-w/--overwrite
(optional): Whether to overwrite existing reduction directories.
Testing
This project uses pytest
. To run the entire test suite:
python -m pytest
To run individual or groups of test files, pass pytest
a path or glob pattern, respectively:
python -m pytest tests/test_cumulative.py
python -m pytest **/test_*_metrics.py
To run a single test, pass its name to the -k
flag:
python -m pytest -k test_precision_recall_curve
Consult the pytest
docs for more details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tensorboard-reducer-0.1.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d15b60824d03d5242eb04a46ad189b3275eb2c66001ac47f8933e237e28adbd8 |
|
MD5 | c53e2b8cbefa5d7d497cf8d7f6241456 |
|
BLAKE2b-256 | f382cbb14254373c9edc86fd23281971f875e0f29697159cad41456502b3a7a2 |
Hashes for tensorboard_reducer-0.1.2-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ffe1b5511e26038cbd8ed90d32a587a066cfc4caec3fb38703786aa8f72efa52 |
|
MD5 | 849ff8032ddf8dcccb904c6c1834404b |
|
BLAKE2b-256 | d7f256f3eb17298dc851793ac5203a5ace597a55537314d41048716dda8bca8c |