Skip to main content

precise

ci License: MIT Python 3.9+

Online (incremental) covariance and correlation estimation — the streaming complement to sklearn.covariance, whose estimators are batch-only and have no partial_fit. Pure Python + numpy; no other required dependencies. And because no estimator wins everywhere, precise also assesses an estimate and recommends one for your data (see Assess & recommend).

📖 Docs: precise.microprediction.org

pip install precise

Use

from precise import EwaCovariance

est = EwaCovariance(r=0.05)
for y in stream:            # y is a 1d observation; pass a 2d array for a batch
    est.partial_fit(y)

est.covariance_             # (n, n) ndarray
est.correlation_            # unit-diagonal correlation
est.precision_             # inverse covariance
est.location_              # running mean
est.fit(X)                 # sklearn-style batch drop-in (X is 2d)

Every estimator is truly online — a constant amount of work per observation, no growing buffers. State is a plain dict, so you can checkpoint mid-stream with get_state() / set_state().

Estimators

Class What it does
EmpiricalCovariance running sample covariance (Welford)
EwaCovariance exponentially weighted (recency-biased)
AdaptiveEwaCovariance EWMA whose forgetting rate speeds up on regime change
LedoitWolfCovariance online Ledoit-Wolf shrinkage towards a scaled identity
OASCovariance online Oracle Approximating Shrinkage (often better-conditioned than LW)
ShrunkCovariance fixed-intensity shrinkage to identity or a constant-correlation target
NonlinearShrinkageCovariance analytical nonlinear spectrum shrinkage — each eigenvalue moved separately, sample eigenvectors kept
WindowedNonlinearShrinkageCovariance the same map over a rolling window — forgets, and equal weights keep the asymptotics exact
EwaNonlinearShrinkageCovariance the same map over an exponentially weighted sample — approximate, but no window boundary to echo shocks
PartialMomentsCovariance exponentially weighted partial-moment (semi-)covariance
HuberCovariance online robust estimator that downweights outliers
TylerCovariance recursive Tyler M-estimator — robust correlation/shape for elliptical data
GeodesicEwaCovariance recency-weighted update along the affine-invariant SPD geodesic
DCCCovariance dynamic conditional correlation — decouples volatility from correlation
FactorCovariance online low-rank + diagonal (approximate factor model); O(d·k) per step
from precise import all_estimators, estimator_from_name
all_estimators()                          # the list of classes (a bake-off in one loop)
estimator_from_name("LedoitWolfCovariance")

Keyed / dynamic universes (river-style)

In streaming/finance settings observations arrive as dicts keyed by name, and the set of names can change over time. keyed(...) decorates any of the estimators above to consume keyed dicts (river-style update / learn_one) and emit keyed output:

from precise import keyed, EwaCovariance

d = keyed(EwaCovariance(r=0.05), dynamic=True)   # changing universe (DynamicUniverse)
d.update({"AAPL": 0.01, "MSFT": -0.02})
d.update({"MSFT": 0.00, "NVDA": 0.03})           # AAPL leaves, NVDA enters
d.covariance_                                     # dict-of-dicts over the live universe
d.to_frame()                                      # pandas DataFrame  (pip install precise[pandas])

k = keyed(EwaCovariance(r=0.05))                  # fixed universe, imputes missing keys (FixedUniverse)

dynamic=False (the default) gives a FixedUniverse (one wrapped estimator, missing keys imputed); dynamic=True gives a DynamicUniverse (a wrapped estimator per live key-set). Both work with any positional estimator — the adapter adds no covariance math of its own.

Composing volatility × correlation

H = D R D is a composition, not a fixed algorithm. ConditionalCovariance lets you pick the per-series volatility model and the correlation estimator independently — DCCCovariance is just the EWMA/EWMA special case:

from precise import ConditionalCovariance, EwaCovariance, LedoitWolfCovariance

est = ConditionalCovariance(vol=EwaCovariance(r=0.02),       # any estimator, used per series in 1-D
                            corr=LedoitWolfCovariance(r=0.05))  # correlation from any estimator

The volatility model can also be any univariate model from microprediction/skaters (Holt, Hosking, …) via from_skater — precise doesn't depend on it; the adapter is duck-typed:

import skaters
from precise import ConditionalCovariance, from_skater
est = ConditionalCovariance(vol=from_skater(skaters.holt), corr=EwaCovariance(r=0.05))

Assess & recommend

No estimator wins everywhere, so precise treats judging and choosing an estimate as first-class alongside producing one.

from precise import all_assessors, suggest

all_assessors()             # scoring rules: LogLikelihood, BlockPseudoLikelihood, SchurLikelihood,
                            # SteinLoss, FrobeniusToTruth, GMVVariance, ... (higher = better)
suggest(X, top=3)           # recommend estimator classes from observable features of X

suggest maps truth-free features of your data (p/n, effective rank, sphericity, condition number, off-diagonal mass, excess kurtosis) to an estimator, via a frozen, numpy-only decision tree. The Schur pseudo-likelihood — a one-parameter (γ) bridge between the full and block-diagonal Gaussian likelihoods — is both an assessor here and the subject of a working paper.

Related

Migrating from precise < 1.0 (the functional "skater" API)? See MIGRATING.md.

Disclaimer

Not investment advice. Just code, subject to the MIT License.

Release files for precise 1.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for precise 1.1.0
File Size Uploaded
precise-1.1.0.tar.gz 44.5 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for precise 1.1.0
File Interpreter ABI Platform
precise-1.1.0-py3-none-any.whl Python 3 none any Details

Total release size: 104.0 kB

Release files / precise-1.1.0.tar.gz

Download URL precise-1.1.0.tar.gz
Size 44.5 kB
Tags Source
SHA-256 checksum
How to use checksums
5da946c7768897afae4d24c7eae19887772088ff1166a8f34e0ae40eda2b60e9
BLAKE2b-256 checksum
How to use checksums
85fc91c773c1cef6b514f8c02a0d65352fc22b4b7b86928369932ce521c0c40a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.16

Release files / precise-1.1.0-py3-none-any.whl

Download URL precise-1.1.0-py3-none-any.whl
Size 59.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
dd38dcd76cbf34be520934c73e280589cfd505029f8642d3e0de0ca043fd85d0
BLAKE2b-256 checksum
How to use checksums
312a6605334af487e139532014a6122549212983c4e8ed87cc2c34e275a4c368
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.16

Release history Release notifications | RSS feed

This release

1.1.0 This release

2 release files

1.0.0

2 release files

0.16.7

2 release files

0.14.1

2 release files

0.14.0

2 release files

0.13.5

2 release files

0.13.2

2 release files

0.13.0

2 release files

0.12.8

2 release files

0.12.7

2 release files

0.12.5

2 release files

0.12.2

2 release files

0.12.1

2 release files

0.12.0

2 release files

0.11.9

2 release files

0.11.8

2 release files

0.11.7

2 release files

0.10.4

2 release files

0.10.3

2 release files

0.10.1

2 release files

0.10.0

2 release files

0.9.5

2 release files

0.9.1

2 release files

0.8.7

2 release files

0.8.1

2 release files

0.8.0

2 release files

0.7.5

2 release files

0.7.3

2 release files

0.7.1

2 release files

0.7.0

2 release files

0.6.1

2 release files

0.5.21

2 release files

0.5.16

2 release files

0.5.11

2 release files

0.5.9

2 release files

0.5.7

2 release files

0.5.5

2 release files

0.4.18

2 release files

0.4.0

2 release files

0.3.1

2 release files

0.1.0

2 release files

0.0.2

2 release files

0.0.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page