enstat

Hallmark feature

enstat is a library to compute ensemble statistics without storing the entire ensemble in memory. In particular, it allows you to compute:

Ensemble averages (and their variance);
Ensemble averages (and their variance) based on a certain binning of a quantity;
Histograms of an ensemble.

Below you find a quick-start. For more information, see the documentation.

Ensemble average

The key feature is to store the sum of the first and second statistical moments and the number of samples. This gives access to the mean (and variance) at all times, while you can keep adding samples.

Suppose that we have 100 realisations, each with 1000 'items', and we want to know the ensemble average of each item:

import enstat
import numpy as np

ensemble = enstat.static()

for realisation in range(100):
    sample = np.random.random(1000)
    ensemble += sample

print(ensemble.mean())

which will print a list of 1000 values, each around 0.5.

This is the equivalent of

import numpy as np

container = np.empty((100, 1000))

for realisation in range(100):
    sample = np.random.random(1000)
    container[realisation, :] = sample

print(np.mean(container, axis=0))

The key difference is that enstat only requires you to have 4 * N values in memory for a sample of size N: the sample itself, the sums of the first and second moment, and the normalisation. Instead the solution with the container uses much more memory.

A nice feature is also that you can keep adding samples to ensemble. You can even store it and continue later.

Ensemble histogram

Same example, but now we want the histogram for predefined bins:

import enstat
import numpy as np

bin_edges = np.linspace(0, 1, 11)
hist = enstat.histogram(bin_edges=bin_edges)

for realisation in range(100):
    sample = np.random.random(1000)
    hist += sample

print(hist.p)

which prints the probability density of each bin (so list of values around 0.1 for these bins).

The histogram class contains two additional nice features.

It has several bin algorithms that NumPy does not have.

It can be used for plotting with an ultra-sort interface, for example:

import enstat
import numpy as np
import matplotlib.pyplot as plt

data = np.random.random(1000)
hist = enstat.histogram.from_data(data, bins=10, mode="log")

fig, ax = plt.subplots()
ax.plot(hist.x, hist.p)
plt.show()

You can even use ax.plot(*hist.plot).

Average per bin

Suppose you have some time series (t) with multiple observables (a and b); e.g.;

import enstat
import numpy as np

t = np.linspace(0, 10, 100)
a = np.random.normal(loc=5, scale=0.1, size=t.size)
b = np.random.normal(loc=1, scale=0.5, size=t.size)

Now suppose that you want to compute the average a, b, and t based on a certain binning of t:

bin_edges = np.linspace(0, 12, 12)
binned = enstat.binned.from_data(t, a, b, names=["t", "a", "b"]m bin_edges=bin_edges)
print(binned["a"].mean())

Installation

Using conda
```
conda install -c conda-forge enstat
```
Using PyPi
```
python -m pip install enstat
```

Disclaimer

This library is free to use under the MIT license. Any additions are very much appreciated. As always, the code comes with no guarantee. None of the developers can be held responsible for possible mistakes.

Metadata

Release files for enstat 0.9.7

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for enstat 0.9.7
File	Size	Uploaded	Provenance
enstat-0.9.7.tar.gz	22.7 kB	Sep 8, 2023	Without provenance	Details

Built distribution (wheel)

Table of built distributions (wheels) for enstat 0.9.7
File	Interpreter	ABI	Platform	Reset
enstat-0.9.7-py3-none-any.whl 13.8 kB Sep 8, 2023 Without provenance	Python 3	none	any	Details

Total release size: 36.5 kB

Release files / enstat-0.9.7.tar.gz

Download URL	enstat-0.9.7.tar.gz
Size	22.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`b5584669d692b59fec9b2456a7525dc1b0d1c415a54148655bf6a63ec8c4a610`
BLAKE2b-256 checksum How to use checksums	`d56aeb1285b5acf049d5d733974fd17cb78657d352a0270b7984e3143bf0dda3`
Upload date	Sep 8, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/4.0.2 CPython/3.11.5`

Release files / enstat-0.9.7-py3-none-any.whl

Download URL	enstat-0.9.7-py3-none-any.whl
Size	13.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`ab02e36c85185317a1d68a9db6f15aaeda6ae3a2e829fab3a529498b1258341c`
BLAKE2b-256 checksum How to use checksums	`c8c125948f277636e40da14275e5c5267c5120f069e708628a28e7c209ad3e41`
Upload date	Sep 8, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/4.0.2 CPython/3.11.5`

Security for enstat 0.9.7

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.