polars-cv

ℹ️ Note: This is a largely AI developed project and still in its early stages. Use at your own discretion.

A Polars plugin for high-performance vision and array operations.

Features

• Modular Pipelines: Define image processing pipelines and apply them to DataFrame columns.
• Expression Arguments: Any parameter that doesn't fix the output shape takes a Polars expression — numeric values, individual list elements, and non-structural enums/flags — on both the pipeline and the geometry namespaces. A null in one of those columns raises by default, or nulls only the rows that depend on it with .on_null_param("null").
• Zero-Copy Performance: Efficient memory management with stride-aware operations.
• Multi-Domain: Seamlessly move between images, geometry (contours), and numeric results.

Installation

pip install polars-cv

Quick Start

import polars as pl
from polars_cv import Pipeline

# Define a pipeline and apply it to a column
pipe = Pipeline().source("image_bytes").resize(height=224, width=224).grayscale()

df = pl.DataFrame({"image": [img1_bytes, img2_bytes]})
result = df.with_columns(
    processed=pl.col("image").cv.pipe(pipe).sink("numpy")
)

Scaling to Large Datasets

Each .cv.pipe(...) call spreads its rows over the plugin's thread pool (sized by POLARS_MAX_THREADS), so a plain eager DataFrame.with_columns(...) uses every core. For data larger than memory, run through the lazy streaming engine, which processes the column in morsels and can spill intermediate state to disk:

result = (
    df.lazy()
    .with_columns(processed=pl.col("image").cv.pipe(pipe).sink("blob"))
    .collect(engine="streaming")
)

The compiled graph is cached process-wide, so per-morsel overhead is just a hash lookup. (The detection-metrics APIs already collect with engine="streaming" internally.)

The plugin's pool and Polars' own are separate and both sized by POLARS_MAX_THREADS, so a query with many plugin expressions can briefly run up to about twice that many threads. If such a query shows a load average far above the core count and erratic timings, lower POLARS_MAX_THREADS — see Streaming & Scaling.

Source Behavior (Auto DType)

source() defaults to "auto", which infers the decode path from the input column's dtype: String → file_path, List/Array → list/array, and Binary → blob when it carries the VIEW protocol magic, image_bytes otherwise. Pass an explicit format to override it.

image_bytes and file_path sources decode image format and dtype at runtime:

• PNG/JPEG usually decode as u8
• 16-bit PNG decodes as u16
• TIFF may decode as u8, u16, f32, or f64

This means the pipeline dtype starts as auto for these sources unless you pin it with dtype=... or an operation that determines dtype (such as normalize, threshold, or cast).

# Runtime decode with automatic dtype
auto_pipe = Pipeline().source("image_bytes").resize(height=224, width=224)

# Pin expected dtype at source (runtime cast when needed)
typed_pipe = Pipeline().source("image_bytes", dtype="f32").resize(height=224, width=224)

When using sink("list") or sink("array"), dtype must be known at planning time. For image_bytes / file_path, choose one of:

• set dtype in source(...)
• add .cast("...")
• use a dtype-fixing operation before the sink

pipe = Pipeline().source("file_path", dtype="f32").resize(height=224, width=224)

result = df.with_columns(values=pl.col("path").cv.pipe(pipe).sink("list"))

Dynamic Pipelines

Use Polars expressions for per-row parameter values:

pipe = (
    Pipeline()
    .source("image_bytes")
    .resize(height=pl.col("target_h"), width=pl.col("target_w"))
    .crop(top=pl.col("crop_y"), left=pl.col("crop_x"), height=100, width=100)
)

df = pl.DataFrame({
    "image": [img1_bytes, img2_bytes],
    "target_h": [224, 256],
    "target_w": [224, 256],
    "crop_x": [10, 20],
    "crop_y": [5, 15],
})

result = df.with_columns(
    processed=pl.col("image").cv.pipe(pipe).sink("numpy")
)

A parameter column may contain nulls. By default that fails the query, naming the column and row; .on_null_param("null") yields null for the affected rows instead — only the outputs that actually depend on the parameter, leaving genuine decode and execution errors raising as before. For a fallback value, fill it in the expression: pl.col("target_h").fill_null(224).

Operations

• Image: resize, resize_scale, resize_to_height, resize_to_width, resize_max, resize_min, thumbnail, grayscale, blur, threshold, crop, rotate, pad, letterbox, flip_h, flip_v.
• Color: convert_color, to_hsv, to_lab, to_bgr, to_ycbcr.
• Channels: channel_select, channel_swap.
• Intensity: adjust_contrast, adjust_gamma, adjust_brightness, invert.
• Convolution & Edge Detection: convolve2d, sobel, laplacian, sharpen, canny.
• Morphology: erode, dilate, morphology_open, morphology_close, morphology_gradient.
• Affine Transforms: warp_affine, shear, rotate_and_scale.
• Enhancement: equalize_histogram.
• Compute: normalize, scale, clamp, relu, cast.
• Layout: transpose, reshape.
• Geometry: extract_contours, rasterize, area, perimeter, centroid, bounding_box.
• Points: normalize, translate, scale, rotate, distance, manhattan_distance, distance_to_contour, signed_distance_to_contour, nearest_point_on_contour, angle_to, midpoint, interpolate, within_bbox.
• Bounding Boxes: pairwise_iou, correspond (via .bbox namespace).
• Analysis: histogram, perceptual_hash, extract_shape, label_reduce.
• Reductions: reduce_sum, reduce_mean, reduce_std, reduce_max, reduce_min, reduce_argmax, reduce_argmin, reduce_percentile, reduce_popcount.
• Metadata: .cv.width(), .cv.height(), .cv.channels(), .cv.image_dtype().
• Byte access: .cv.read_bytes() — read a path column's bytes (local or cloud) without decoding.
• Display: show_images() for Jupyter notebook visualization.
• Detection Metrics: Precision-Recall, AP, mAP, FROC, LROC, F1, confusion matrix, bootstrap confidence intervals — all weighted by image_metadata.weight.

Detection Metrics

One call from predictions and ground truth (one row per object: boxes, polygons or instance masks) to a COCO-style report:

from polars_cv.metrics import evaluate_detections

report = evaluate_detections(
    preds, gts, geometry=("x1", "y1", "x2", "y2"), box_format="xyxy"
)
report.summary       # map, map_50, map_75, mar
report.per_class     # AP and recall per class
report.ci("map")     # bootstrap interval, images resampled

evaluate_heatmaps does the same for probability maps against masks (with a FROC summary), and evaluate_segmentation gives Dice, IoU and surface distances per image. Every number is a Statistic, so bootstrap_ci gives it an interval.

The lower layers stay one step away. Matchers build a DetectionTable, and every metric reads it:

from polars_cv.metrics import PreMatchedAdapter, precision_recall_curve, average_precision

# Wrap pre-matched detection data. `image_meta` (one row per evaluated slide,
# with image_id + n_gts) defines the population, so slides with no detections
# are not dropped from the denominators.
adapter = PreMatchedAdapter()
table = adapter.match(
    df,
    pred_col="confidence",
    gt_col="is_tp",
    image_id_col="slide_id",
    image_meta=slides,
)

# Compute metrics
pr = precision_recall_curve(table)
ap = average_precision(table)

print(f"AP: {ap:.3f}")
print(pr.curve)  # score, precision, recall, cum_tp, cum_fp, cum_weighted_tp, cum_weighted_fp

Every metric is weighted by the table's image_metadata.weight (scikit-learn sample_weight semantics: a weight of k counts an image k times; unit weights give the plain counts). The bootstrap intervals re-estimate sample-derived weights in every replicate by stratifying the resample on weight cells, so reweighting a study to a target mix needs no extra step; weight_scheme="fixed" carries known (design or continuous) weights instead.

Available matchers: match_detections (object tables of any geometry), ContourMatcher (heatmap/mask), BBoxMatcher (bounding boxes), PreMatchedAdapter (pre-computed TP/FP).

Available metrics: precision_recall_curve, average_precision, mean_average_precision, froc_auc, lroc_auc, froc_curve_lazy, lroc_curve_lazy, confusion_at_threshold, precision_at_threshold, recall_at_threshold, f1_at_threshold, and the lazy, group-aware bootstrap intervals bootstrap_ci (any Statistic: AP, mean_ap(), FROCAUC, …), froc_auc_ci_lazy, lroc_auc_ci_lazy, average_precision_ci_lazy.

For full details, see the Documentation