batchstats 0.6

Efficient batch statistics computation library for Python.

BatchStats

BatchStats computes statistics on data that arrives in batches, so you can stream or process large datasets without loading everything into memory. Feed batches with update_batch, then call the object to get the final result.

Installation

pip install batchstats

Or with conda/mamba:

conda install -c conda-forge batchstats

Quick Start

import numpy as np
from batchstats import BatchMean, BatchVar

data_stream = (np.random.randn(100, 10) for _ in range(10))

batch_mean = BatchMean()
batch_var = BatchVar()

for batch in data_stream:
    batch_mean.update_batch(batch)
    batch_var.update_batch(batch)

mean = batch_mean()
variance = batch_var()

print(f"Mean shape: {mean.shape}")
print(f"Variance shape: {variance.shape}")

Available Statistics

• BatchSum / BatchNanSum
• BatchWeightedSum
• BatchMean / BatchNanMean
• BatchWeightedMean
• BatchMin / BatchNanMin
• BatchMax / BatchNanMax
• BatchPeakToPeak / BatchNanPeakToPeak
• BatchVar
• BatchStd
• BatchCov
• BatchCorr

Docs: https://batchstats.readthedocs.io