Skip to main content

polars-stats

polars-stats is a Polars expression plugin that exposes scipy.stats-style probability distributions natively inside Polars expressions:

  • Lazy-native: every method returns a pl.Expr, so a distribution composes inside a LazyFrame query under the optimiser, with no materialisation.

  • Column-valued parameters: any distribution parameter can be a scalar or a Polars expression. A single instance describes a different distribution per row:

    import polars as pl
    import polars_stats as ps
    
    norm = ps.Normal(mu=pl.col("mu"), sigma=pl.col("sigma"))
    norm.cdf(pl.col("x"))
    
  • Polars null and error semantics: a null input gives a null result, and an invalid parameter raises a ComputeError rather than silently returning NaN.

  • Reproducible sampling: every draw is keyed on (seed, row index), so a seeded column repeats across runs, chunkings, thread counts, and both engines.

scipy already does the per-row maths: stats.norm(loc=mu_array, scale=sigma_array).sf(x_array) broadcasts parameter arrays and scores every element against its own distribution, vectorised, with no Python loop. The difference is where the result lands. scipy returns a NumPy array, so a LazyFrame has to collect() first, pushdown stops at that boundary, and realigning the result through later joins and filters is your problem. Here it stays a pl.Expr the planner can see. Why polars-stats has the full comparison.

The math runs in Rust on top of the statrs crate; the Python layer is a thin, typed surface of distribution classes.

Why

Statistical work in Polars today means falling back to .to_pandas() / .to_numpy(), then reaching for one of:

  • scipy.stats, which exits the lazy engine and materialises everything,
  • Python UDFs via map_elements, which are slow and hold the GIL,
  • hand-rolled per-distribution expressions, which are ad hoc and error-prone.

The row-varying, vectorised, lazy-native case is what polars-stats targets.

Quick example

Anomaly scoring, where each row carries its own baseline:

import polars as pl
import polars_stats as ps

readings = pl.LazyFrame(
    {
        "value": [9.8, 101.0, 12.1, 250.0],
        "mu": [10.0, 100.0, 10.0, 100.0],
        "sigma": [0.5, 2.0, 0.5, 2.0],
    }
)

norm = ps.Normal(mu="mu", sigma="sigma")

anomalies = readings.with_columns(upper_tail=norm.sf("value")).filter(pl.col("upper_tail") < 0.01).collect()
print(anomalies)
shape: (2, 4)
┌───────┬───────┬───────┬────────────┐
│ value ┆ mu    ┆ sigma ┆ upper_tail │
│ ---   ┆ ---   ┆ ---   ┆ ---        │
│ f64   ┆ f64   ┆ f64   ┆ f64        │
╞═══════╪═══════╪═══════╪════════════╡
│ 12.1  ┆ 10.0  ┆ 0.5   ┆ 0.000013   │
│ 250.0 ┆ 100.0 ┆ 2.0   ┆ 0.0        │
└───────┴───────┴───────┴────────────┘

Each row is scored against its own Normal(mu, sigma), in one vectorised pass, without leaving the lazy engine.

Numerical accuracy

The maths runs on statrs, and make audit sweeps every method against an mpmath oracle at 50 digits, including inputs many decades past where scipy itself saturates. For tail work on Normal, LogNormal, Gamma, Poisson and the closed-form distributions (Uniform, Exponential, Cauchy, Pareto, Weibull, Bernoulli, Geometric, DiscreteUniform), use log_cdf / log_sf rather than the linear pair, and isf(q) rather than ppf(1 - q). Beta and Binomial inherit several documented statrs-side limits in this release: there the log methods underflow with the linear ones, and the extreme lower tail of ppf misbehaves. Every known limit is listed with a regime and a magnitude in Numerical accuracy.

Installation

pip install polars-stats

Runtime needs polars>=1.15 and Python >=3.10.

Wheels are abi3, so one wheel per platform serves every Python from 3.10 up: manylinux x86_64 and aarch64, macOS x86_64 and arm64, Windows x64, plus an sdist. Alpine (musl) and Windows on ARM have no wheel today even though polars publishes one for both; there pip falls back to the sdist and needs a Rust toolchain. Open an issue if you need either target and it gets added.

Documentation

Full docs at fbruzzesi.github.io/polars-stats: the API reference with the distribution catalogue and method surface, and the architecture and design notes.

A note on the Rust code

I am not a Rust expert, and a good part of the Rust layer was written with AI assistance.

What I vouch for is the behaviour, which is pinned by an extensive test suite: parity against scipy.stats on every method, property-based invariants, and bit-identity between the constant-parameter fast paths and the general per-row paths.

Treat the Rust idioms with the appropriate skepticism: if you spot something that should be written differently, an issue or PR is very welcome.

polars-stats is not the first take on statistics inside Polars expressions. Three projects cover neighbouring ground, and if your need matches their scope they may serve you well:

  • polars-random generates random columns as native Polars expressions (uniform, normal, binomial, integers), with column-valued parameters, per-call seeds, and a global set_random_seed. It registers .random namespaces on Expr, DataFrame, and LazyFrame, which reads very naturally when sampling is the whole job. Its focus is sampling; polars-stats treats sampling as one method of a full distribution object, next to pdf / cdf / sf / ppf, their numerically stable log variants, and closed-form moments.
  • polars_rng exposes one sampling expression per distribution (prng.normal(mu=pl.col("x"), sigma=3)), also as a Rust plugin over the same statrs crate, also with column-valued parameters. Its sampling catalogue is wider than what polars-stats ships (Laplace, plus categorical and integer draws), so for pure simulation it may be the better fit. The differences are scope and reproducibility: it is sampling only, with no pdf / cdf / ppf or moments, and it draws from a thread-local RNG with no seed argument, where polars-stats keys every draw on (seed, row index) so a seeded column repeats across runs, chunkings, and engines.
  • polars_normal_stats covers the Normal distribution through three focused expressions, normal_cdf / normal_ppf / normal_pdf, each evaluated at a column of points. Its mean and std travel as plugin kwargs, so they are scalars: the common case, handled in three functions and nothing more. polars-stats generalises the same idea to a catalogue of distributions behind one scipy-like class API, passes parameters as plugin inputs so they can be columns, and adds survival functions, log_cdf / log_sf, and reproducible sampling.

License

This project is licensed under the MIT license.

Metadata

Release files for polars-stats 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for polars-stats 0.2.0
File Size Uploaded Provenance Actions
polars_stats-0.2.0.tar.gz 478.4 kB With provenance Details for polars_stats-0.2.0.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for polars-stats 0.2.0
File Actions
polars_stats-0.2.0-cp310-abi3-win_amd64.whl CPython 3.10 abi3 Windows x86-64 Details for polars_stats-0.2.0-cp310-abi3-win_amd64.whl
polars_stats-0.2.0-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl CPython 3.10 abi3 Linux glibc 2.17+ x86-64 Details for polars_stats-0.2.0-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
polars_stats-0.2.0-cp310-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl CPython 3.10 abi3 Linux glibc 2.17+ ARM64 Details for polars_stats-0.2.0-cp310-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl
polars_stats-0.2.0-cp310-abi3-macosx_11_0_arm64.whl CPython 3.10 abi3 macOS 11.0+ ARM64 Details for polars_stats-0.2.0-cp310-abi3-macosx_11_0_arm64.whl
polars_stats-0.2.0-cp310-abi3-macosx_10_12_x86_64.whl CPython 3.10 abi3 macOS 10.12+ x86-64 Details for polars_stats-0.2.0-cp310-abi3-macosx_10_12_x86_64.whl

Total release size: 33.0 MB

Release files / polars_stats-0.2.0.tar.gz

Download URL polars_stats-0.2.0.tar.gz
Size 478.4 kB
Tags Source
SHA-256 checksum
How to use checksums
cbd690b2a3b7966dd522e199e0d1c919354fd18a98ed29a787acc68dcfa301a8
BLAKE2b-256 checksum
How to use checksums
e168a06ed5234f95fb7a3ff7ce59418e034a514322c9c6d0c4449cc353590d74
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 8, 2026.

Transparency log

Release files / polars_stats-0.2.0-cp310-abi3-win_amd64.whl

Download URL polars_stats-0.2.0-cp310-abi3-win_amd64.whl
Size 7.7 MB
Tags CPython 3.10 Windows x86-64 abi3
SHA-256 checksum
How to use checksums
8fd6ee8459849b3a6692311ee6598034764c7856a04a38a77082f62819cba6af
BLAKE2b-256 checksum
How to use checksums
4d694f788f6d2364835a0791d98c125b54b56060f85824cf001e8b1db1711131
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 8, 2026.

Transparency log

Release files / polars_stats-0.2.0-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl

Download URL polars_stats-0.2.0-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Size 6.7 MB
Tags CPython 3.10 Linux glibc 2.17+ x86-64 abi3
SHA-256 checksum
How to use checksums
f0e149cb3f8879017e0a2a3778d320f85917ebc99d9969d6e6d9a9f42094dc7d
BLAKE2b-256 checksum
How to use checksums
3459498c72cc0effd34e693dc83f340c968053200fbe5b9d375f36b3bec777ab
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 8, 2026.

Transparency log

Release files / polars_stats-0.2.0-cp310-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl

Download URL polars_stats-0.2.0-cp310-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl
Size 5.8 MB
Tags CPython 3.10 Linux glibc 2.17+ ARM64 abi3
SHA-256 checksum
How to use checksums
ea9160d8bb0bd31085fda74757c4bdc22e0a87397306b14917982910c9b765f0
BLAKE2b-256 checksum
How to use checksums
8809c371d2e23c47e289ce00fdde7f773d0472ecb2c1a9b918a09302ab683905
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 8, 2026.

Transparency log

Release files / polars_stats-0.2.0-cp310-abi3-macosx_11_0_arm64.whl

Download URL polars_stats-0.2.0-cp310-abi3-macosx_11_0_arm64.whl
Size 5.7 MB
Tags CPython 3.10 abi3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
394da2626b0225ddf409200e7573d3fea0a83443dab4c5949a961f1a2c7ca4b6
BLAKE2b-256 checksum
How to use checksums
916f3d5b9b51086ebf00ed729fe0d79beb7c35954ae89ab10670220b4201b2c2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 8, 2026.

Transparency log

Release files / polars_stats-0.2.0-cp310-abi3-macosx_10_12_x86_64.whl

Download URL polars_stats-0.2.0-cp310-abi3-macosx_10_12_x86_64.whl
Size 6.5 MB
Tags CPython 3.10 abi3 macOS 10.12+ x86-64
SHA-256 checksum
How to use checksums
47aa9fbd4473117e2e3fd51cd4ce2db714144a83271500ccb5c181df141a98eb
BLAKE2b-256 checksum
How to use checksums
74c2a1a52fdb29cd42e76af5b4b5b2663f105f294b89351445caa9e1f2969434
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 8, 2026.

Transparency log

Security for polars-stats 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

All 6 files in this release have been uploaded with provenance metadata.

Select from 6 files

Attestation for polars_stats-0.2.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 8, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page