Skip to main content

covmats

License Stars Python PyPI Downloads Build Status Documentation Status Coverage codacy Precommit: enabled Ruff Checked with ty DOI

🐍 Covariance matrices representation.

The complete and up to date documentation can be found here: https://covmats.readthedocs.io.

🎯 Motivations

Calculations involving covariance matrices (e.g. linear algebra, data whitening, multivariate normal function evaluation) are often performed more efficiently using a decomposition of the covariance matrix instead of the covariance matrix itself. For large scale application, a dense covariance matrix would not even fit in memory and one must rely on low-rank approximations. This package allows the user to construct an object representing a covariance matrix using any of several decompositions/approximations and perform calculations using a common interface.

The common interface CovarianceMatrix can be seen as a extension of the class scipy.stats.Covariance as it inherits from it (thus making it compatible with all scipy.stats functions and classes) and dope it with LinearOperator capabilities.

The package is used in large-scale inversion packages such as pypcga, pyesmda and pyrtid.

🚀 Quick start

To install covmats, the easiest way is through pip:

pip install covmats

Or alternatively using conda

conda install covmats

You might also clone the repository and install from source

pip install -e .

Once the installation is done, covmats is straightforward to use and proposes the following full-rank covariance representations:

It also provides low-rank approximations suitable for large scale problems:

as well as kernel-based, matrix-free linear operators for point-cloud data:

and a small hierarchy of prior/drift terms for geostatistical regularization:

The two companion tutorial notebooks, examples_covariances.py and examples_priors.py, walk through every one of these classes in detail. The rest of this section gives a condensed overview.

Let’s start by importing numpy, scipy and covmats for the tests and define a random number generator seed for reproducibility:

import numpy as np
import scipy as sp
import covmats

rng_seed = 2026

First example with a diagonal covariance matrix

In the following, we define a (3 x 3) covariance matrix defining only its diagonal (i.e., all elements of the random vectors are independent). Using scipy, it is possible to compute the pdf.

d = [1, 2, 3]
A33 = np.diag(d)  # a diagonal covariance matrix
x = [4, -2, 5]  # a point of interest
dist = sp.stats.multivariate_normal(mean=[0, 0, 0], cov=A33)
dist.pdf(x)
np.float64(4.9595685102808205e-08)

It is possible to obtain a dense representation in a straightforward manner:

cov_diag33 = covmats.CovViaDiagonal(d)
dist = sp.stats.multivariate_normal(mean=[0, 0, 0], cov=cov_diag33)
dist.pdf(x)
np.float64(4.9595685102808205e-08)

It is compatible with the stats API from scipy since the base class inherit from Covariance.

Every representation also exposes todense, solve, precision and log_pdet, as well as sample_mvnormal, whiten and colorize for fast Monte-Carlo sampling and data whitening:

samples = cov_diag33.sample_mvnormal(shape=[10000], random_state=rng_seed)
np.round(np.cov(samples, rowvar=False), 1)
array([[ 1.,  0., -0.],
       [ 0.,  2., -0.],
       [-0., -0.,  3.]])

Cholesky and precision-based representations

CovViaCholesky wraps a dense Cholesky factor L such that A = L @ L.T, while CovViaPrecisionCholesky does the same for the precision matrix Q = A^-1 (the natural representation for Gaussian Markov Random Fields, where Q is sparse). Sparse counterparts, CovViaSparseCholesky and CovViaSparsePrecisionCholesky, build on a SparseCholeskyFactor (an LDL’ factorization) for large, sparse problems:

rng = np.random.default_rng(rng_seed)
n = 4
B = rng.random((n, n))
A = B @ B.T + n * np.eye(n)  # a random SPD matrix
cov_cho = covmats.CovViaCholesky(np.linalg.cholesky(A))
np.allclose(cov_cho.todense(), A)
True

Low-rank representations

For very large problems, CovViaEigenFactorization (a truncated eigen decomposition) and CovViaEnsemble (an ensemble of anomalies, as used in ensemble Kalman filtering) never require forming the dense covariance matrix:

ensemble = rng.multivariate_normal(np.zeros(6), np.eye(6) * 2 + 0.3, size=200)
cov_ens = covmats.CovViaEnsemble(ensemble)
cov_ens.shape
(6, 6)

get_linop_eigen_factorization and eigen_factorize_cov_mat build a randomized low-rank eigen factorization directly from any CovarianceMatrix or CovKernelAsLinop instance, and get_explained_var reports how much variance each retained mode captures – useful to pick a truncation rank.

Kernel-based linear operators

When the covariance is defined analytically through a kernel evaluated on a point cloud, CovKernelAsLinop (dense evaluation, any point cloud) and CovKernelAsLinopViaFFT (FFT-based, regular grids only) expose a LinearOperator without requiring the covariance matrix to be assembled up front:

pts = covmats.get_pts_coords_regular_grid(mesh_dim=1.0, shape=(6, 6))
cov_kernel = covmats.CovKernelAsLinop(
    pts, lambda d: np.exp(-d), len_scale=np.array([2.0, 2.0])
)
cov_kernel.shape
(36, 36)

Priors and drift matrices

Alongside covariance representations, covmats provides PriorTerm subclasses to describe the deterministic mean/trend of a field, from a simple NullPriorTerm to a LinearDriftMatrix expressing a spatially-varying trend as m = X @ beta:

drift = covmats.LinearDriftMatrix(pts)
drift.beta = np.array([1.0, 0.5, -0.5])
drift.get_values(np.zeros(pts.shape[0])).shape
(36,)

🏗️ Complete example with supporting paper coming Q1 2026.

🔑 License

This project is released under the BSD 3-Clause License.

Copyright (c) 2026, Antoine COLLET. All rights reserved.

For more details, see the LICENSE file included in this repository.

⚠️ Disclaimer

This software is provided “as is”, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, or non-infringement. In no event shall the authors or copyright holders be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the software or the use or other dealings in the software.

By using this software, you agree to accept full responsibility for any consequences, and you waive any claims against the authors or contributors.

📧 Contact

For questions, suggestions, or contributions, you can reach out via:

We welcome contributions!

📚 References

TODO

  • Free software: SPDX-License-Identifier: BSD-3-Clause

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

covmats-0.3.1.tar.gz (2.8 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

covmats-0.3.1-py3-none-any.whl (2.8 MB view details)

Uploaded Python 3

File details

Details for the file covmats-0.3.1.tar.gz.

File metadata

  • Download URL: covmats-0.3.1.tar.gz
  • Upload date:
  • Size: 2.8 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.11

File hashes

Hashes for covmats-0.3.1.tar.gz
Algorithm Hash digest
SHA256 7f8e6f6df3d9158371aeeb2095f04bba8068b7313679f7a41dab3c7dd4cc0778
MD5 bebf4cbb055d72d9670366fdc4cb0491
BLAKE2b-256 df4fedfc806eeccb661184eca8a17bec9382854f169d85ab5daa6cd7012b47a4

See more details on using hashes here.

File details

Details for the file covmats-0.3.1-py3-none-any.whl.

File metadata

  • Download URL: covmats-0.3.1-py3-none-any.whl
  • Upload date:
  • Size: 2.8 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.11

File hashes

Hashes for covmats-0.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 4fa436b9fc904fce76de4df00a80c39d8bff421b34c42a48d4a4f5544ec61de1
MD5 da84c80fa8947dfc8625032ddfadae80
BLAKE2b-256 34c9623b67d70fd7bed7621b70b6a46c42a615f880649bff35a75ba50b9cce51

See more details on using hashes here.

Release history Release notifications | RSS feed

0.3.2

2 files

This release

0.3.1 This release

2 files

0.3.0

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page