polars-cv 0.38.0

Image processing as Polars expressions: decode, transform and analyse images in a DataFrame column, in Rust, on every core, without a Python loop.

_{Every panel is computed by polars-cv from a CC0 photo (scripts/make_readme_figure.py). equalize_histogram() is a contrast stretch: it remaps intensities so they spread evenly over 0–255.}

Install

pip install polars-cv      # or: uv add polars-cv

Wheels for Linux (x86_64, aarch64) and macOS (Apple silicon), CPython 3.11+, Polars 2.0+. Other platforms build from source with Rust 1.96+.

Quickstart

A Pipeline describes the work, .cv.pipe() applies it to a column, and .sink() picks the output format.

from pathlib import Path

import polars as pl
from polars_cv import Pipeline

df = pl.DataFrame({"path": sorted(str(p) for p in Path("photos").glob("*.jpg"))})

pipe = Pipeline().source("file_path").resize(height=224, width=224).grayscale()
out = df.with_columns(gray=pl.col("path").cv.pipe(pipe).sink("numpy"))

Sources include image_bytes, local paths and s3:///gs:///az:///http(s):// URLs. Sinks include numpy, array, list, png and jpeg. See Sources and Pipelines.

One decode, several outputs, per-row parameters

A folder of photos to centre-cropped 224×224 training tensors plus JPEG thumbnails. Each file is decoded once and feeds both outputs, and the crop box is computed per row from each image's header.

from pathlib import Path

import polars as pl
from polars_cv import Pipeline

df = pl.DataFrame({"path": sorted(str(p) for p in Path("photos").glob("*.jpg"))})
df = df.with_columns(pl.col("path").cv.image_info().struct.unnest())  # header only

side = pl.min_horizontal("width", "height")  # a per-row value
img = pl.col("path").cv.pipe(Pipeline().source("file_path", dtype="u8"))
x = img.pipe(
    Pipeline()
    .crop(top=(pl.col("height") - side) // 2, left=(pl.col("width") - side) // 2,
          height=side, width=side)
    .resize(height=224, width=224, filter="bilinear")
).alias("x")
thumb = img.pipe(Pipeline().resize_max(128)).alias("thumb")

out = df.with_columns(
    o=x.merge_pipe(thumb).sink({
        "x": ("array", {"shape": [224, 224, 3]}),
        "thumb": ("jpeg", {"quality": 85}),
    })
).unnest("o")
batch = out["x"].to_numpy()  # (n_images, 224, 224, 3) uint8

Any parameter that does not change the output's rank, dtype or domain accepts a Polars expression. See Multi-output pipelines.

Branch, rejoin, optimise: cutting out the foreground

Pipelines compose into a graph. This one cuts a cup and saucer out of a photo with a transparent background: two cues find the china (bright pixels, and the saturated red that the brown table lacks), their union is traced to an outline, and the outline is rasterized back into an alpha channel for the photo.

from pathlib import Path

import polars as pl
from polars_cv import Pipeline

base = Pipeline().source("file_path").resize_max(512)
img = pl.col("path").cv

# Two cues, each written as if it stood alone. The threshold is a column.
bright = img.pipe(base.grayscale().threshold(190))
saturated = img.pipe(
    base.to_hsv().channel_select(1).threshold(pl.col("min_saturation"))
)

# Rejoin: either cue, cleaned up and traced; keep the largest outline's hull.
outline = bright.maximum(saturated).pipe(
    Pipeline().morphology_open(ksize=5).extract_contours().largest().convex_hull()
)

# The outline as an anti-aliased alpha channel joined to the photo's colours.
photo = img.pipe(base)
alpha = outline.rasterize(shape=photo, supersample=4)
cutout = photo.channel_merge(alpha).alias("cutout")

expr = cutout.merge_pipe(outline.area().alias("area"))
paths = sorted(str(p) for p in Path("photos").glob("*.jpg"))
out = (
    pl.LazyFrame({"path": paths, "min_saturation": 210})
    .with_columns(o=expr.sink({"cutout": "png", "area": "native"}))
    .unnest("o")
    .collect()
)

expr.sink(..., return_expr=False).show_graph() draws the graph that runs (it needs polars-cv[viz] and Graphviz's dot):

Written out, this is eight nodes with three decodes; it runs as six with one. The optimiser computes each distinct value once: the three pipelines built from base share one read and resize (the SOURCE node), and each linear run of steps that nothing else reads becomes a single node, so the brightness threshold runs straight through maximum to the hull in one. Pass opt_flags=False to sink to see the graph as written; the outputs are byte for byte the same. The whole graph is one Polars expression: it runs in the lazy and streaming engines, per row, with min_saturation read from each row's column, and returns the PNG and the outline's area together.

Why polars-cv

• Images are a column. Paths, bytes, labels, metadata and results sit in one DataFrame, so filtering, joining and group_by on image properties is plain Polars, not a second data structure kept in sync with a list of arrays.
• Polars runs it. Pipelines are expressions: they run lazily, in the streaming engine for larger-than-memory data, with each call's rows spread over every core by Rust code. There is no Python per image.
• Parameters can be expressions. Crop boxes from annotations, sizes from metadata, thresholds from a lookup, evaluated per row.

Benchmarks

5,000 COCO val2017 images on 4 cores, against PIL, OpenCV (process pool), torchvision (DataLoader), Daft and Hugging Face datasets. Each is written the way its users would write it and checked against PIL's output before timing.

polars-cv trails OpenCV on thumbnails because its JPEG encoder is slower than libjpeg-turbo. Methodology, every table and the command to reproduce them are in benchmarks/.

What's included

The full list is in the operations guide and the API reference; the drawing guide builds the logo above from these ops.

Larger-than-memory data

Eager calls already use every core. For data that does not fit in memory, use a LazyFrame: .collect() streams by default, and .collect_batches() hands you one batch at a time.

from pathlib import Path

import polars as pl
from polars_cv import Pipeline

pipe = Pipeline().source("file_path", dtype="u8").resize(height=64, width=64)
lf = pl.LazyFrame({"path": sorted(str(p) for p in Path("photos").glob("*.jpg"))})
for batch in (
    lf.select(x=pl.col("path").cv.pipe(pipe).sink("array", shape=[64, 64, 3]))
    .collect_batches(chunk_size=2)
):
    x = batch["x"].to_numpy()  # (rows, 64, 64, 3) uint8

Thread counts, morsel sizing and memory budgets are covered in Streaming & Scaling.

Project status

ℹ️ Note: This is a largely AI-developed project and still in its early stages. Use at your own discretion.

polars-cv is 0.x: a minor release may break the API. Breaking changes are listed in the changelog, each with a migration guide. Pin a minor version (polars-cv~=0.38.0) if you depend on it. Outputs are checked against NumPy, SciPy, OpenCV and imagehash in CI.

Contributing

Issues and pull requests are welcome. CONTRIBUTING.md covers the build, and scripts/verify.sh runs every check CI runs.

Citation

If you use polars-cv in research, please cite the version you used:

@software{dar_polars_cv,
  author = {Dar, Hesham},
  title  = {polars-cv: Image processing as Polars expressions},
  url    = {https://github.com/heshamdar/polars-cv},
  year   = {2026}
}

Licensed under either of MIT or Apache-2.0, at your option.