polars-distr

Native Polars expressions for statistics over piecewise-uniform distributions. The plugin computes means, CDF values, quantiles, and exact CRPS inside lazy and streaming query plans without Python UDFs.

Install with uv add polars-distr or pip install polars-distr.

import polars as pl
import polars_distr as pld  # registers the Distribution dtype

result = (
    pl.scan_parquet("distributions.parquet")
    .select(
        pld.quantiles("distribution", [0.1, 0.5, 0.9]).alias("quantiles"),
        pld.crps("distribution", "observation").alias("crps"),
    )
    .collect(engine="streaming")
)

The .distr accessor

Importing the package also registers a .distr expression namespace that delegates to these same functions:

pl.col("weights").distr.quantile(0.5, edges="edges")
pl.struct(weights="weights", edges="edges").distr.crps("observation")

It adds no semantics. It exists for callers that can only reach registered Polars namespaces and cannot import this package — notably YAML/JSON pipeline configs whose expression strings are evaluated with pl alone in scope. The subject is a Distribution column or raw Struct{weights, edges} storage; passing edges instead treats the subject as the weights, so a config never has to spell out pl.struct(...).

Named quantiles

Pass quantiles() a sequence of levels for a fixed-size Array in request order, or a name-to-level mapping for a Struct in mapping order — the same single traversal either way:

result = frame.select(
    pld.quantiles("distribution", {"p10": 0.1, "p50": 0.5, "p90": 0.9}).alias("q")
).unnest("q")  # -> p10, p50, p90 columns

Values and ordering are identical target for target. Validity differs in shape only: where the sequence form nulls the outer Array row, the mapping form yields a non-null Struct whose every field is null, so the unnested columns keep the row and carry the null per target.

Constructing distributions

distribution() accepts either paired fixed-width Array columns or paired variable-width List columns:

distribution = pld.distribution("weights", "edges")

It produces the logical Distribution dtype with serialized identity polars_distr.piecewise_uniform.v1. Its storage is a Struct with exactly weights and edges, in that order. Fixed storage must be Array/Array with edge width equal to weight width plus one. Ragged storage must be List/List. Mixed layouts are rejected by the constructor and statistics kernels.

Integer, Float32, and Float64 child values are supported. Invalid distribution rows produce null results; statically invalid schemas raise during schema resolution. Raw paired Struct storage is also accepted by every statistics kernel for interoperability.

v0.2 breaking changes

• The public vocabulary is distribution, weights, and edges; the v0.1 histogram and counts names are removed.
• The extension identity is polars_distr.piecewise_uniform.v1. Data written with the pre-contract polars_distr.distribution.v1 identity is not guaranteed to load.
• distribution() accepts paired Array/Array and paired List/List storage. Mixed Array/List storage is rejected.