polars-list-math

polars-list-math is a Python/Rust package with list, JSON, and URL expression helpers for Polars.

Release notes and upgrade guidance are recorded in CHANGELOG.md.

Import the package once to register extra methods on Expr.list and Expr.str:

import polars as pl
import polars_list_math  # noqa: F401

For type-checked code, import the top-level expression helpers described below.

Install

pip install polars-list-math
# or
uv add polars-list-math

Requires Python 3.12+ and polars>=1.39.3.

Methods

Top-level expression helpers provide the statically typed API for Pyright, Pylance, and other type checkers:

The Python helper py_list_similarity(...) computes weighted similarity for plain Python sequences. py_list_containment(...) computes directed set containment for Python sequences, ignoring ranks and duplicates.

The Expr.list and Expr.str methods are registered dynamically and remain available as convenient runtime syntax. Python type checkers cannot safely augment Polars' own namespace classes from another installed package, so use the equivalent top-level functions in code checked by Pylance or Pyright.

If a future Polars release ships native methods with the same names, this package leaves the native implementation untouched.

Quick Examples

from polars_list_math import (
    list_combinations,
    list_mean_similarity,
    list_similarity,
    list_zip,
)

df = pl.DataFrame(
    {
        "a": [[1, 2, 3]],
        "b": [[2, 1, 3]],
        "groups": [[[1, 2, 3], [1, 2, 3], [4, 5, 6]]],
    }
)

df.with_columns(
    list_similarity("a", "b").alias("similarity"),
    list_mean_similarity("groups").alias("mean_similarity"),
    list_combinations("a").alias("pairs"),
    list_zip("a", "b", fields=["a", "b"]).alias("zipped"),
)

Convert JSON strings to nested Polars values. String values stay strings, JSON null stays null, and other values become compact JSON strings. Invalid JSON returns null by default; pass strict=True to raise instead.

from polars_list_math import json_array_values, json_object_items

json_df = pl.DataFrame(
    {
        "object": ['{"name":"Ada","active":true,"note":null}'],
        "array": ['["python",42,true,null]'],
    }
)

json_df.select(
    json_object_items("object").alias("items"),
    json_array_values("array").alias("values"),
)

Encode query parameters and build URLs from expressions:

from polars_list_math import url_build, url_query_encode

pages = pl.DataFrame(
    {
        "host": ["example.com"],
        "path": ["/search"],
        "term": ["rust polars"],
        "page": [2],
    }
)

query = url_query_encode(pl.struct(q=pl.col("term"), page=pl.col("page")))

pages.select(
    url_build(
        scheme=pl.lit("https"),
        netloc="host",
        path="path",
        query=query,
    ).alias("url")
)

More complete, runnable examples are available in the examples directory.

Nested zip builder

ZipBuilder combines nested list columns with zip options and ordered eval/filter operations at each level. build() returns a normal Polars expression:

from polars_list_math import ZipBuilder

expr = (
    ZipBuilder("a", "b")
    .level(pad=True)
    .filter(pl.element().struct.field("a").is_not_null())
    .build("nested_zip")
)

The constructor configures the outer zip; each level() adds one inner zip. See the builder guide for transformations, broadcasting, and deeper nesting.

Performance experiments

Reproducible performance experiments live in the performance directory. Each experiment has its own README with the tested scenario, run instructions, environment, and recorded results.

The current dataframe_construction experiment compares construction of 30-column Polars DataFrames from a regular dataclass, a slotted dataclass, and a named tuple, including nested structs and lists.

uv run pytest performance/dataframe_construction -v -s

Development

make install
make develop
make test

Build and check the package locally:

make check-dist

Release steps are in docs/publishing.md.