polars-cv

ℹ️ Note: This is a largely AI developed project and still in its early stages. Use at your own discretion.

A Polars plugin for high-performance vision and array operations.

Features

• Modular Pipelines: Define image processing pipelines and apply them to DataFrame columns.
• Expression Arguments: Any parameter that doesn't fix the output shape takes a Polars expression — numeric values, individual list elements, and non-structural enums/flags — on both the pipeline and the geometry namespaces. A null in one of those columns raises by default, or nulls only the rows that depend on it with .on_null_param("null").
• Zero-Copy Performance: Efficient memory management with stride-aware operations.
• Multi-Domain: Seamlessly move between images, geometry (contours), and numeric results.

Installation

pip install polars-cv

Quick Start

import polars as pl
from polars_cv import Pipeline

# Define a pipeline and apply it to a column
pipe = Pipeline().source("image_bytes").resize(height=224, width=224).grayscale()

df = pl.DataFrame({"image": [img1_bytes, img2_bytes]})
result = df.with_columns(
    processed=pl.col("image").cv.pipe(pipe).sink("numpy")
)

Scaling to Large Datasets

Each .cv.pipe(...) call spreads its rows over the plugin's thread pool (sized by POLARS_MAX_THREADS), so a plain eager DataFrame.with_columns(...) uses every core. For data larger than memory, run through the lazy streaming engine (a lazy .collect()'s default since Polars 2.0), which processes the column in morsels and spills intermediate state to disk:

result = (
    df.lazy()
    .with_columns(processed=pl.col("image").cv.pipe(pipe).sink("blob"))
    .collect()
)

The compiled graph is cached process-wide, so per-morsel overhead is just a hash lookup. (The detection-metrics APIs collect with polars' default engine, streaming since Polars 2.0, so they stream unless you set another engine affinity.)

The plugin's pool and Polars' own are separate, and both are sized by POLARS_MAX_THREADS. Threads running polars-cv rows share one budget of POLARS_MAX_THREADS: a call running alone uses the whole pool, and concurrent calls (one per streaming morsel) keep their rows on their own threads once the budget is used. If a query with many plugin expressions still shows a load average far above the core count, lower POLARS_MAX_THREADS — see Streaming & Scaling.

Source Behavior (Auto DType)

source() defaults to "auto", which infers the decode path from the input column's dtype: String → file_path, List/Array → list/array, and Binary → blob when it carries the VIEW protocol magic, image_bytes otherwise. Pass an explicit format to override it.

image_bytes and file_path sources decode image format and dtype at runtime:

• PNG/JPEG usually decode as u8
• 16-bit PNG decodes as u16
• TIFF may decode as u8, u16, f32, or f64

This means the pipeline dtype starts as auto for these sources unless you pin it with dtype=... or an operation that determines dtype (such as normalize, threshold, or cast).

# Runtime decode with automatic dtype
auto_pipe = Pipeline().source("image_bytes").resize(height=224, width=224)

# Pin expected dtype at source (runtime cast when needed)
typed_pipe = Pipeline().source("image_bytes", dtype="f32").resize(height=224, width=224)

When using sink("list") or sink("array"), dtype must be known at planning time. For image_bytes / file_path, choose one of:

• set dtype in source(...)
• add .cast("...")
• use a dtype-fixing operation before the sink

pipe = Pipeline().source("file_path", dtype="f32").resize(height=224, width=224)

result = df.with_columns(values=pl.col("path").cv.pipe(pipe).sink("list"))

Dynamic Pipelines

Use Polars expressions for per-row parameter values:

pipe = (
    Pipeline()
    .source("image_bytes")
    .resize(height=pl.col("target_h"), width=pl.col("target_w"))
    .crop(top=pl.col("crop_y"), left=pl.col("crop_x"), height=100, width=100)
)

df = pl.DataFrame({
    "image": [img1_bytes, img2_bytes],
    "target_h": [224, 256],
    "target_w": [224, 256],
    "crop_x": [10, 20],
    "crop_y": [5, 15],
})

result = df.with_columns(
    processed=pl.col("image").cv.pipe(pipe).sink("numpy")
)

A parameter column may contain nulls. By default that fails the query, naming the column and row; .on_null_param("null") yields null for the affected rows instead — only the outputs that actually depend on the parameter, leaving genuine decode and execution errors raising as before. For a fallback value, fill it in the expression: pl.col("target_h").fill_null(224).

Operations

• Image: resize, resize_scale, resize_to_height, resize_to_width, resize_max, resize_min, thumbnail, grayscale, blur, threshold, crop, rotate, pad, pad_to_size, letterbox, flip, flip_h, flip_v.
• Color: convert_color, to_hsv, to_lab, to_bgr, to_ycbcr.
• Channels: channel_select, channel_swap.
• Intensity: adjust_contrast, adjust_gamma, adjust_brightness, invert.
• Convolution & Edge Detection: convolve2d, sobel, laplacian, sharpen, canny.
• Morphology: erode, dilate, morphology_open, morphology_close, morphology_gradient.
• Affine Transforms: warp_affine, shear, rotate_and_scale.
• Enhancement: equalize_histogram.
• Compute: normalize, scale, add_constant, subtract_constant, clamp, clamp_min, clamp_max, relu, cast.
• Element-wise math: abs, neg, sign, sqrt, square, reciprocal, round, floor, ceil, trunc.
• Binary (on LazyPipelineExpr, between two pipelines): add, subtract, multiply, divide, blend, maximum, minimum, bitwise_and, bitwise_or, bitwise_xor, apply_mask, apply_contour_mask, channel_merge.
• Layout: transpose, reshape, assert_shape.
• Geometry: extract_contours, rasterize, simplify, convex_hull, scale_contour, largest, area, perimeter, centroid, bounding_box; the .contour namespace adds iou, dice, hausdorff_distance, contains_point, boundary_distances and more.
• Points: normalize, translate, scale, rotate, distance, manhattan_distance, distance_to_contour, signed_distance_to_contour, nearest_point_on_contour, angle_to, midpoint, interpolate, within_bbox.
• Bounding Boxes: pairwise_iou, correspond (via .bbox namespace).
• Analysis: histogram, perceptual_hash, extract_shape, label_reduce.
• Reductions: reduce_sum, reduce_mean, reduce_std, reduce_max, reduce_min, reduce_argmax, reduce_argmin, reduce_percentile, reduce_popcount, and statistics() for several in one pass.
• Metadata: .cv.width(), .cv.height(), .cv.channels(), .cv.image_dtype().
• Byte access: .cv.read_bytes() — read a path column's bytes (local or cloud) without decoding.
• Display: show_images() for Jupyter notebook visualization.
• Detection Metrics: Precision-Recall, AP, mAP, FROC, LROC, F1, confusion matrix, bootstrap confidence intervals — all weighted by image_metadata.weight.

Detection Metrics

One call from predictions and ground truth (one row per object: boxes, polygons or instance masks) to a COCO-style report:

from polars_cv.metrics import evaluate_detections

report = evaluate_detections(
    preds, gts, geometry=("x1", "y1", "x2", "y2"), box_format="xyxy"
)
report.summary       # map, map_50, map_75, mar
report.per_class     # AP and recall per class
report.ci("map")     # bootstrap interval, images resampled

evaluate_heatmaps does the same for probability maps against masks (with a FROC summary), and evaluate_segmentation gives Dice, IoU and surface distances per image. Every number is a Statistic, so bootstrap_ci gives it an interval.

The lower layers stay one step away. Matchers build a DetectionTable, and every metric reads it:

from polars_cv.metrics import PreMatchedAdapter, precision_recall_curve, average_precision

# Wrap pre-matched detection data. `image_meta` (one row per evaluated slide,
# with image_id + n_gts) defines the population, so slides with no detections
# are not dropped from the denominators.
adapter = PreMatchedAdapter()
table = adapter.match(
    df,
    pred_col="confidence",
    gt_col="is_tp",
    image_id_col="slide_id",
    image_meta=slides,
)

# Compute metrics
pr = precision_recall_curve(table)
ap = average_precision(table)

print(f"AP: {ap:.3f}")
print(pr.curve)  # score, precision, recall, cum_tp, cum_fp, cum_weighted_tp, cum_weighted_fp

Every metric is weighted by the table's image_metadata.weight (scikit-learn sample_weight semantics: a weight of k counts an image k times; unit weights give the plain counts). The bootstrap intervals re-estimate sample-derived weights in every replicate by stratifying the resample on weight cells, so reweighting a study to a target mix needs no extra step; weight_scheme="fixed" carries known (design or continuous) weights instead.

Available matchers: match_detections (object tables of any geometry), ContourMatcher (heatmap/mask), BBoxMatcher (bounding boxes), PreMatchedAdapter (pre-computed TP/FP).

Available metrics: precision_recall_curve, average_precision, mean_average_precision, froc_auc, lroc_auc, froc_curve_lazy, lroc_curve_lazy, confusion_at_threshold, precision_at_threshold, recall_at_threshold, f1_at_threshold, and the lazy, group-aware bootstrap intervals bootstrap_ci (any Statistic: AP, mean_ap(), FROCAUC, …), froc_auc_ci_lazy, lroc_auc_ci_lazy, average_precision_ci_lazy.

For full details, see the Documentation