Skip to main content

covmats

License Stars Python PyPI Downloads Build Status Documentation Status Coverage codacy Precommit: enabled Ruff Checked with ty DOI

🐍 Covariance matrices representation.

The complete and up to date documentation can be found here: https://covmats.readthedocs.io.

🎯 Motivations

Calculations involving covariance matrices (e.g. linear algebra, data whitening, multivariate normal function evaluation) are often performed more efficiently using a decomposition of the covariance matrix instead of the covariance matrix itself. For large scale application, a dense covariance matrix would not even fit in memory and one must rely on low-rank approximations. This package allows the user to construct an object representing a covariance matrix using any of several decompositions/approximations and perform calculations using a common interface.

The common interface CovarianceMatrix can be seen as a extension of the class scipy.stats.Covariance as it inherits from it (thus making it compatible with all scipy.stats functions and classes) and dope it with LinearOperator capabilities.

The package is used in large-scale inversion packages such as pypcga, pyesmda and pyrtid.

🚀 Quick start

To install covmats, the easiest way is through pip:

pip install covmats

Or alternatively using conda

conda install covmats

You might also clone the repository and install from source

pip install -e .

Once the installation is done, covmats is straightforward to use and proposes the following full-rank covariance representations:

It also provides low-rank approximations suitable for large scale problems:

as well as kernel-based, matrix-free linear operators for point-cloud data:

and a small hierarchy of prior/drift terms for geostatistical regularization:

The two companion tutorial notebooks, examples_covariances.py and examples_priors.py, walk through every one of these classes in detail. The rest of this section gives a condensed overview.

Let’s start by importing numpy, scipy and covmats for the tests and define a random number generator seed for reproducibility:

import numpy as np
import scipy as sp
import covmats

rng_seed = 2026

First example with a diagonal covariance matrix

In the following, we define a (3 x 3) covariance matrix defining only its diagonal (i.e., all elements of the random vectors are independent). Using scipy, it is possible to compute the pdf.

d = [1, 2, 3]
A33 = np.diag(d)  # a diagonal covariance matrix
x = [4, -2, 5]  # a point of interest
dist = sp.stats.multivariate_normal(mean=[0, 0, 0], cov=A33)
dist.pdf(x)
np.float64(4.9595685102808205e-08)

It is possible to obtain a dense representation in a straightforward manner:

cov_diag33 = covmats.CovViaDiagonal(d)
dist = sp.stats.multivariate_normal(mean=[0, 0, 0], cov=cov_diag33)
dist.pdf(x)
np.float64(4.9595685102808205e-08)

It is compatible with the stats API from scipy since the base class inherit from Covariance.

Every representation also exposes todense, solve, precision and log_pdet, as well as sample_mvnormal, whiten and colorize for fast Monte-Carlo sampling and data whitening:

samples = cov_diag33.sample_mvnormal(shape=[10000], random_state=rng_seed)
np.round(np.cov(samples, rowvar=False), 1)
array([[ 1.,  0., -0.],
       [ 0.,  2., -0.],
       [-0., -0.,  3.]])

Cholesky and precision-based representations

CovViaCholesky wraps a dense Cholesky factor L such that A = L @ L.T, while CovViaPrecisionCholesky does the same for the precision matrix Q = A^-1 (the natural representation for Gaussian Markov Random Fields, where Q is sparse). Sparse counterparts, CovViaSparseCholesky and CovViaSparsePrecisionCholesky, build on a SparseCholeskyFactor (an LDL’ factorization) for large, sparse problems:

rng = np.random.default_rng(rng_seed)
n = 4
B = rng.random((n, n))
A = B @ B.T + n * np.eye(n)  # a random SPD matrix
cov_cho = covmats.CovViaCholesky(np.linalg.cholesky(A))
np.allclose(cov_cho.todense(), A)
True

Low-rank representations

For very large problems, CovViaEigenFactorization (a truncated eigen decomposition) and CovViaEnsemble (an ensemble of anomalies, as used in ensemble Kalman filtering) never require forming the dense covariance matrix:

ensemble = rng.multivariate_normal(np.zeros(6), np.eye(6) * 2 + 0.3, size=200)
cov_ens = covmats.CovViaEnsemble(ensemble)
cov_ens.shape
(6, 6)

get_linop_eigen_factorization and eigen_factorize_cov_mat build a randomized low-rank eigen factorization directly from any CovarianceMatrix or CovKernelAsLinop instance, and get_explained_var reports how much variance each retained mode captures – useful to pick a truncation rank.

Kernel-based linear operators

When the covariance is defined analytically through a kernel evaluated on a point cloud, CovKernelAsLinop (dense evaluation, any point cloud) and CovKernelAsLinopViaFFT (FFT-based, regular grids only) expose a LinearOperator without requiring the covariance matrix to be assembled up front:

pts = covmats.get_pts_coords_regular_grid(mesh_dim=1.0, shape=(6, 6))
cov_kernel = covmats.CovKernelAsLinop(
    pts, lambda d: np.exp(-d), len_scale=np.array([2.0, 2.0])
)
cov_kernel.shape
(36, 36)

Priors and drift matrices

Alongside covariance representations, covmats provides PriorTerm subclasses to describe the deterministic mean/trend of a field, from a simple NullPriorTerm to a LinearDriftMatrix expressing a spatially-varying trend as m = X @ beta:

drift = covmats.LinearDriftMatrix(pts)
drift.beta = np.array([1.0, 0.5, -0.5])
drift.get_values(np.zeros(pts.shape[0])).shape
(36,)

🏗️ Complete example with supporting paper coming Q1 2026.

🔑 License

This project is released under the BSD 3-Clause License.

Copyright (c) 2026, Antoine COLLET. All rights reserved.

For more details, see the LICENSE file included in this repository.

⚠️ Disclaimer

This software is provided “as is”, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, or non-infringement. In no event shall the authors or copyright holders be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the software or the use or other dealings in the software.

By using this software, you agree to accept full responsibility for any consequences, and you waive any claims against the authors or contributors.

📧 Contact

For questions, suggestions, or contributions, you can reach out via:

We welcome contributions!

📚 References

TODO

  • Free software: SPDX-License-Identifier: BSD-3-Clause

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

covmats-0.3.2.tar.gz (2.8 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

covmats-0.3.2-py3-none-any.whl (2.8 MB view details)

Uploaded Python 3

File details

Details for the file covmats-0.3.2.tar.gz.

File metadata

  • Download URL: covmats-0.3.2.tar.gz
  • Upload date:
  • Size: 2.8 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.11

File hashes

Hashes for covmats-0.3.2.tar.gz
Algorithm Hash digest
SHA256 2cd9707a0eb10de0adaf812b6065cba1111a4f950b2084c4d79a37948e6d2e2f
MD5 8b56a9bdfd1f9db4c4cc92ffcd104ab5
BLAKE2b-256 5b2a798f0a74b1522c3051cdc70c0489851a1e8fd7b8ec2fa5a8da1bc640033c

See more details on using hashes here.

File details

Details for the file covmats-0.3.2-py3-none-any.whl.

File metadata

  • Download URL: covmats-0.3.2-py3-none-any.whl
  • Upload date:
  • Size: 2.8 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.11

File hashes

Hashes for covmats-0.3.2-py3-none-any.whl
Algorithm Hash digest
SHA256 70b6c8a2c7d8bb2dd24a63828c85ac05b1a1126b15e15a4cc35d9dec148457e5
MD5 18015376a3d20be10e8b0b3557189976
BLAKE2b-256 7f9275fac1ce4212390d31ffbe4ce444d3dc61ae3b4cf1dbbf5ec743d844a655

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

0.3.2 This release

2 files

0.3.1

2 files

0.3.0

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page