tensorboard-reducer 0.1.6

Reduce multiple TensorBoard runs to new event (or CSV) files

This project was inspired by tensorboard-aggregator (similar project built with TensorFlow rather than PyTorch) and this SO answer.

Compute reduced statistics (mean, std, min, max, median or any other numpy operation) of multiple TensorBoard runs matching a directory glob pattern. This can for instance be used when training multiple identical models to reduce the noise in their loss/accuracy/error curves to establish statistical significance in performance improvements. The aggregation results can be saved to disk either as new TensorBoard event files or in CSV format.

Requires PyTorch and TensorBoard. No TensorFlow installation required.

Installation

pip install tensorboard-reducer

Usage

Example:

tb-reducer -i 'glob_pattern/of_dirs_to_reduce*' -o basename_of_output_dir -r mean,std,min,max

tb-reducer has the following flags:

• -i/--indirs-glob (required): Glob pattern of the run directories to reduce.
• -o/--outdir (required): Name of the directory to save the new reduced run data. If --format is tb-events, a separate directory will be created for each reduce op (mean, std, ...) suffixed by the op's name (outdir-mean, outdir-std, ...). If --format is csv, a single file will created and outdir must end with a .csv extension.
• -r/--reduce-ops (optional): Comma-separated names of numpy reduction ops (mean, std, min, max, ...). Default is mean. Each reduction is written to a separate outdir suffixed by its op name, e.g. if outdir='my-new-run, the mean reduction will be written to my-new-run-mean.
• -f/--format: Output format of reduced TensorBoard runs. One of tb-events for regular TensorBoard event files or csv. If csv, -o/--outdir must have .csv extension and all reduction ops will be written to a single CSV file rather than separate directories for each reduce op. Use pandas.read_csv("path/to/file.csv", header=[0, 1], index_col=0) to read data back into memory as a multi-index dataframe.
• -w/--overwrite (optional): Whether to overwrite existing outdirs/CSV files.