Skip to main content

Welford

Python (Numpy) implementation of Welford’s algorithm, which is online or parallel algorithm for calculating variances.

The algorithm is described in the followings,

Welford’s method is more numerically stable than the standard method. The theoritical background of Welford’s method is mentioned in detail on the following blog articles. Please refer them if you are interested in.

This library is inspired by the jvf’s implementation, which is implemented without using numpy library.

Install

Download package via PyPI repository

$ pip install welford

Example

For Online Calculation

import numpy as np
from welford import Welford

# Initialize Welford object
w = Welford()

# Input data samples sequentialy
w.add(np.array([0, 100]))
w.add(np.array([1, 110]))
w.add(np.array([2, 120]))

# output
print(w.mean)  # mean --> [  1. 110.]
print(w.var_s)  # sample variance --> [1, 100]
print(w.var_p)  # population variance --> [ 0.6666 66.66]

# You can add other samples after calculating variances.
w.add(np.array([3, 130]))
w.add(np.array([4, 140]))

# output with added samples
print(w.mean)  # mean --> [  2. 120.]
print(w.var_s)  # sample variance --> [  2.5 250. ]
print(w.var_p)  # population variance --> [  2. 200.]

Welford object supports initialization with data samples and batch addition of samples.

# Initialize Welford object with samples
ini = np.array([[0, 100],
                [1, 110],
                [2, 120]])
w = Welford(ini)

# output
print(w.mean)  # mean --> [  1. 110.]
print(w.var_s)  # sample variance --> [1, 100]
print(w.var_p)  # population variance --> [ 0.66666667 66.66666667]

# add other samples through batch method
other_samples = np.array([[3, 130],
                          [4, 140]])
w.add_all(other_samples)

# output with added samples
print(w.mean)  # mean --> [  2. 120.]
print(w.var_s)  # sample variance --> [  2.5 250. ]
print(w.var_p)  # population variance --> [  2. 200.]

For Parallel Calculation

Welford also offers parallel calculation method for variance.

import numpy as np
from welford import Welford

# Initialize two Welford objects
w_1 = Welford()
w_2 = Welford()

# Each object will calculate variance of each samples in parallel.
# On w_1
w_1.add(np.array([0, 100]))
w_1.add(np.array([1, 110]))
w_1.add(np.array([2, 120]))
print(w_1.var_s)  # sample variance -->[  1. 100.]
print(w_1.var_p)  # population variance -->[ 0.66666667 66.66666667]

# On w_2
w_2.add(np.array([3, 130]))
w_2.add(np.array([4, 140]))
print(w_2.var_s)  # sample variance -->[ 0.5 50. ]
print(w_2.var_p)  # sample variance -->[ 0.25 25.  ]

# You can Merge objects to get variance of WHOLE samples
w_1.merge(w_2)
print(w.var_s)  # sample variance --> [  2.5 250. ]
print(w_1.var_p)  # sample variance -->[  2. 200.]

Metadata

Release files for welford 0.2.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for welford 0.2.5
File Size Uploaded Provenance
welford-0.2.5.tar.gz 5.1 kB Without provenance Details

Release files / welford-0.2.5.tar.gz

Download URL welford-0.2.5.tar.gz
Size 5.1 kB
Tags Source
SHA-256 checksum
How to use checksums
f0dedc139528c027c0331b0d254b6542bfa0a6782996bd3efe6c7b191ffb5116
BLAKE2b-256 checksum
How to use checksums
114a1074e17e6fa5ee23b18772d570bf90eb90dde5e0b49e5c7e065272b7a6fb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via Python-urllib/3.7

Security for welford 0.2.5

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page