polars-random

Generate random numbers and statistical distributions natively in Polars DataFrames — a NumPy-style random API exposed as first-class Polars expressions, with reproducible seeds and per-row parameters.

polars-random is a Rust plugin offering four equivalent entry points so it composes naturally with the rest of polars:

import polars as pl
import polars_random as pr  # registers DataFrame/LazyFrame/Expr namespaces

# 1. eager Series
pr.normal(mean=0.0, std=1.0, size=5, seed=42)

# 2. as a polars expression in any context
df = pl.DataFrame({"id": range(5)})
df.with_columns(noise=pr.normal(mean=0.0, std=1.0, seed=42))
df.with_columns(noise=pl.col("id").random.normal(seed=42))

# 3. as a DataFrame method (returns a new DataFrame with the column appended)
df.random.normal(mean=0.0, std=1.0, seed=42, name="noise")

# 4. inside a lazy pipeline
df.lazy().random.normal(seed=42, name="noise").collect()

Available distributions: rand / uniform, normal, binomial, randint. Every parameter (low, high, mean, std, n, p) accepts a Python scalar, a column name ("my_col"), or any pl.Expr. Nulls in column-valued parameters propagate as null in the output (no panic).

Why polars-random?

• Polars-native — outputs are regular Polars columns, composable with the rest of your pipeline (no NumPy round-trips).
• Per-row parameters — mean, std, low, high, n, p can come from other columns, so each row can be drawn from a different distribution.
• Reproducible — pass seed=... per call, or set one global seed with pr.set_random_seed(...).
• Fast — implemented in Rust on top of rand / rand_distr.

Installation

uv add polars-random

poetry add polars-random

pip install polars-random

How it works (mental model)

Every distribution follows the same shape:

df.random.<distribution>(<params>, seed=None, name=None)

• <params> are the distribution's parameters (e.g. low/high, mean/std, n/p).
• Each parameter accepts a Python literal, a column name as a string, or a Polars expression (pl.col(...), arithmetic, etc.). Within a single call, all distribution parameters must be the same kind — either all literals or all expressions/column-names (no mixing).
• seed makes the draw reproducible. Omit it for entropy-based randomness.
• name is the new column's name. Defaults to the distribution name ("rand", "normal", "binomial").
• The result is a new pl.DataFrame with the column appended. Calls chain.

Coming from NumPy?

When used as a DataFrame/LazyFrame method or via the pl.col(...).random namespace, the output length is taken from the parent — no size= needed. Use size=N only with the top-level functions for "give me N values without a frame."

Seeding & reproducibility

Every draw takes an optional seed=. There are two ways to make results reproducible:

Per call — pass seed= to a single expression:

pr.normal(mean=0.0, std=1.0, seed=42)

Globally — set one seed once with pr.set_random_seed(...). Any draw that omits seed= then derives its seed from this global generator:

import polars as pl
import polars_random as pr

pr.set_random_seed(42)

df = pl.DataFrame({"id": range(5)})
df.with_columns(
    a=pr.normal(),               # reproducible — no per-call seed needed
    b=pr.rand(),                 # independent of `a`, also reproducible
)

This solves the "keep the distribution definition separate from where I set the seed" workflow: define length-free expressions (pr.normal(3.0, 1.0)) and let one global seed make the whole run reproducible, without threading seed=42 through every call.

Semantics:

• Each expression consumes the global generator once, so distinct random columns in the same query stay independent (they are not byte-for-byte identical), just like NumPy's global RNG or polars' own set_random_seed.
• Re-calling pr.set_random_seed(42) rewinds the sequence, so a script re-run reproduces the same draws.
• An explicit seed= on a call always overrides the global seed for that call.
• Without any global seed, seedless draws use OS entropy (the historical default) — nothing changes for existing code.

Want two identical columns?

Seedless draws under a global seed are always independent (that is the point), so they never coincide. To make two columns equal, give them the same explicit seed= — that pins both to the same draw and bypasses the global generator:

df.with_columns(
    a=pr.normal(seed=7),
    b=pr.normal(seed=7),   # a == b, byte-for-byte
)

Reproducibility depends on call order

Under a global seed, each seedless draw takes the next value from the generator, so reproducibility depends on the order and number of seedless draws — the same sequence of calls reproduces the same columns. Inserting or reordering a seedless draw shifts every later one (exactly like NumPy's or Polars' global RNG). If a specific column must stay fixed regardless of surrounding code, pin it with an explicit seed=.

pr.set_random_seed is independent of polars.set_random_seed. Polars seeds its own operations (.sample(), .shuffle(), …) and exposes only a setter — there is no public way for a plugin to read Polars' seed — so polars-random keeps its own global seed. Set both if you want Polars and polars-random reproducible in the same script.

Distributions

df.random.rand (uniform) · also aliased as df.random.uniform

import polars as pl
import polars_random

df = pl.DataFrame({
    "custom_low":  [0.0, 10.0, 100.0],
    "custom_high": [1.0, 20.0, 200.0],
})

(
    df
    # Scalar parameters
    .random.rand(low=1_000., high=2_000., seed=42, name="rand_scalar")
    # Default range [0, 1)
    .random.rand(seed=42, name="rand_default")
    # Per-row parameters via expression
    .random.rand(low=pl.col("custom_low"), high=pl.col("custom_high"), seed=42, name="rand_expr")
    # Per-row parameters via column name
    .random.rand(low="custom_low", high="custom_high", seed=42, name="rand_str")
)

df.random.normal

import polars as pl
import polars_random

df = pl.DataFrame({
    "custom_mean": [0.0, 5.0, -3.0],
    "custom_std":  [1.0, 2.0, 0.5],
})

(
    df
    .random.normal(mean=3., std=2., seed=42, name="normal_scalar")
    .random.normal(seed=42, name="normal_default")  # mean=0, std=1
    .random.normal(mean=pl.col("custom_mean"), std=pl.col("custom_std"), seed=42, name="normal_expr")
    .random.normal(mean="custom_mean", std="custom_std", seed=42, name="normal_str")
)

df.random.binomial

import polars as pl
import polars_random

df = pl.DataFrame({
    "n": [10, 50, 100],
    "p": [0.1, 0.5, 0.9],
})

(
    df
    .random.binomial(n=100, p=.5, seed=42, name="binomial_scalar")
    .random.binomial(n=pl.col("n"), p=pl.col("p"), seed=42, name="binomial_expr")
    .random.binomial(n="n", p="p", seed=42, name="binomial_str")
)

df.random.randint

Uniform random integers in [low, high) (high is exclusive, matching numpy.random.randint).

df.random.randint(low=0, high=10, seed=42)            # one column, scalar bounds
df.random.randint(low="lo", high="hi", seed=42)       # per-row bounds via columns

Beyond df.random — same kernel, four entry points

import polars as pl
import polars_random as pr

# 1. Top-level: returns a Series of N random values (NumPy-style).
pr.normal(mean=0, std=1, size=1_000, seed=42)

# 2. Top-level inside any expression (length comes from the surrounding context).
df.with_columns(noise=pr.normal(mean=0, std=1, seed=42))

# 3. Expression namespace — anchor random draws to an existing column.
df.with_columns(noise=pl.col("id").random.normal(mean=0, std=1, seed=42))

# 4. LazyFrame — keep random draws inside a lazy plan.
df.lazy().random.binomial(n=10, p=0.5, seed=42, name="trials").collect()

When a parameter is column-valued (pl.col(...), a column name, or any expression) and contains nulls, the output is null at those rows instead of raising.

Benchmarks

polars-random is a Rust plugin built on rand / rand_distr, so on the single-threaded path it matches numpy.random for the distributions where vectorisation dominates (uniform, normal, randint) and is sampler-bound for binomial. Run inside a LazyFrame with the streaming engine, polars parallelises the elementwise plugin across worker threads and pulls substantially further ahead.

Speedup is numpy_best_time / polars_random_best_time (best of 5×2 timed calls); a value above 1 means polars-random is faster. Full table and methodology: benchmarks/results.md.

Eager / lazy in-memory (single-threaded plugin)

df.with_columns(pr.<dist>(..., seed=42)) — the typical "add a random column" shape:

Lazy + streaming engine (parallel)

lf.with_columns(pr.<dist>(..., seed=42)).collect(engine="streaming"):

At small sizes the polars expression engine pays a fixed per-call cost (a few hundred microseconds) so numpy wins; from ~1M rows raw kernel speed dominates. On the streaming engine, binomial — sampler-bound on a single thread — is the biggest winner because the heavy per-row work parallelises cleanly.

Two caveats:

1. Streaming re-seeds per chunk. The streaming engine processes data in chunks and the plugin is invoked once per chunk with the same seed=, so the resulting column is deterministic for a given chunking but differs bit-for-bit from the in-memory engine. Both are valid samples from the distribution; pick the engine first, then fix seed.
2. NumPy → Polars also costs nothing extra at scale — the numpy -> pl.Series row in benchmarks/results.md is within ~1% of plain numpy. The polars-random win is in the kernel itself plus parallelism, not in avoiding a copy.

Reproducing

# release build of the Rust extension
just install-release

# run the benchmark (writes benchmarks/results.md and benchmarks/results.json)
just bench

just bench is equivalent to:

uv run --with numpy python benchmarks/benchmark.py

The script accepts --sizes, --repeats, --inner, --output, and --json flags; see python benchmarks/benchmark.py --help. Defaults are --sizes 10000,100000,1000000,10000000,50000000 --repeats 5 --inner 2.

Documentation

Full API reference: https://diegoglozano.github.io/polars-random/

License

MIT