---

Prepare Polars data for estimators that implement the scikit-learn interface, with reusable feature expressions, column selection, and schema validation. Use these transformers in a sklearn.pipeline.Pipeline, following the same fit-and-apply conventions used by scikit-learn and estimator-style libraries such as XGBoost. Work with eager DataFrame inputs or keep LazyFrame queries deferred until the model needs materialized features.

Put feature preparation inside the pipeline. Define your Polars expressions once with FeatureEngineer, train with pipeline.fit(raw_X, y), and predict with pipeline.predict(new_raw_X). The pipeline generates the features before the model runs, using the same definitions and execution order in both paths. You can keep feature preparation together with the model instead of maintaining separate feature-building functions and remembering to call them before each prediction.

Requires Python 3.11+, Polars 1.20+, and scikit-learn 1.6+. CI runs on Python 3.11 and 3.12, including pandas interoperability tests.

Documentation

Read the complete documentation for the user guide, API reference, runnable examples, troubleshooting, and production guidance. The sources live in docs/index.md; docs/development.md explains local preview and GitHub Pages publishing. The site becomes available after the Documentation workflow deploys successfully.

Why Put Features in the Pipeline?

Feature engineering is part of the model's input preparation. Keeping it in a fitted pipeline gives training and prediction one shared preparation path: validate the raw schema, compute dependent features in order, select or drop columns, and pass the prepared frame to the estimator. New observations arrive as raw data; the pipeline prepares them at prediction time.

You define once	The fitted pipeline applies it
Raw column and dtype requirements	Validate incoming observations before feature preparation.
Feature expressions and their dependencies	Generate the same named features during training and prediction.
Columns to retain or drop	Provide the fitted feature layout to the model.
Additional learned preprocessing	Reuse fitted scaling or other estimator-based preprocessing steps.

This keeps the preparation logic visible alongside the estimator and reduces the chance of omitting a step or using a different feature definition in prediction code. FeatureEngineer executes your expressions and records schema expectations; it does not learn feature statistics. Put transformations that need learned training statistics in additional fitted pipeline steps.

Installation

Install the package into your project:

uv add polars-learn
# Or use pip:
pip install polars-learn

For optional pandas input support:

uv add 'polars-learn[pandas]'
# Or use pip:
pip install 'polars-learn[pandas]'

See Development for running this repository from a checkout.

Choose a Class

Class	Use it when	What fitting does
SchemaSpec	You already know the required column names and dtypes.	Checks your explicit contract and records output names.
SchemaValidator	Training data should define the schema for later input.	Learns every training column's name, order, and dtype.
FeatureSpec	You want to describe one feature operation.	This is a description, not a transformer; there is no fit.
FeatureEngineer	You want to execute feature operations in order.	Validates the transformation and records input/output schemas.
ColumnSelector	You want to keep, reorder, or rename selected columns.	Checks selected columns and records their output names.
FeaturePassthrough	You want an explicit unchanged-column pipeline step or branch.	Checks selected columns and records their unchanged names.

The five transformers implement the scikit-learn estimator interface: fit, transform, fit_transform, get_feature_names_out, get_params, set_params, and set_output. They support cloning and named pipeline parameters, such as features__features or model__max_depth. FeatureSpec is an operation description, not an estimator. All six classes accept verbosity. See the API reference for every constructor option, fitted attribute, and public method.

Quick Start

Define the preparation once and train the complete pipeline on raw input. This example creates notional and log_notional inside the pipeline, removes quantity from the model's feature set, and retains price.

import numpy as np
import polars as pl
from sklearn.ensemble import RandomForestRegressor
from sklearn.pipeline import Pipeline

from polars_learn import FeatureEngineer, FeatureSpec, SchemaValidator

raw_train = pl.DataFrame({"price": [10.0, 20.0, 30.0, 40.0], "quantity": [2, 3, 4, 5]})
y = np.array([1.0, 2.0, 3.0, 4.0])

pipeline = Pipeline(
    [
        ("raw_schema", SchemaValidator()),
        (
            "features",
            FeatureEngineer(
                [
                    FeatureSpec(pl.col("price") * pl.col("quantity"), alias="notional"),
                    FeatureSpec(
                        pl.col("notional").log1p(),
                        alias="log_notional",
                        drop=["quantity"],
                    ),
                ]
            ).set_output(transform="polars"),
        ),
        ("model", RandomForestRegressor(n_estimators=20, random_state=0)),
    ]
)

pipeline.fit(raw_train, y)

# Prediction takes new raw observations; feature preparation runs inside the pipeline.
raw_predict = pl.DataFrame({"price": [15.0, 25.0], "quantity": [2, 4]})
predictions = pipeline.predict(raw_predict)
assert predictions.shape == (2,)
assert np.isfinite(predictions).all()

# Inspect the prepared feature names without building features outside the pipeline.
assert pipeline[:-1].get_feature_names_out().tolist() == [
    "price",
    "notional",
    "log_notional",
]
assert raw_predict.columns == ["price", "quantity"]

The prediction input contains price and quantity; you supply neither notional nor log_notional. pipeline.predict(raw_predict) validates the raw schema, computes the derived columns, drops quantity from the model input, and then predicts. Keep the target y separate from the raw feature frame. Fit the complete pipeline on training input and reuse that fitted pipeline for validation and prediction.

The pipeline expects the fitted raw input contract at prediction time. Do not feed it a frame that has already been feature-engineered, or refit its preprocessing on prediction data. Missing required raw columns and incompatible schemas raise errors according to the configured policies.

The individual transformers also work on their own: fit(X) returns the transformer, transform(X) applies it, and fit_transform(X) fits and prepares the same input. Calling transform or get_feature_names_out before a successful fit raises scikit-learn's NotFittedError. Input frames are never mutated.

Feature Engineering

FeatureEngineer turns an ordered set of Polars expressions into a reusable pipeline step. Each transform evaluates those definitions against the incoming raw frame, so the fitted pipeline can prepare new observations whenever you call predict. The standalone examples below show how to define and inspect that step; in a predictive workflow, include it before the final estimator as in the quick start.

Create a FeatureSpec with one required expression argument: pass a single pl.Expr or a nonempty sequence of expressions. The specification normalizes either form to a tuple without changing your input sequence. Use native expression names, an expression's .alias(...), or the specification's alias argument to name its outputs. An explicit alias requires exactly one expression with one output.

import polars as pl

from polars_learn import FeatureEngineer, FeatureSpec

quotes = pl.DataFrame({"ask": [11.0], "bid": [9.0]})
spec = FeatureSpec(
    expression=[
        (pl.col("ask") - pl.col("bid")).alias("spread"),
        ((pl.col("ask") + pl.col("bid")) / 2).alias("mid"),
    ],
    drop=["ask", "bid"],
    name="quote_features",
)
result = FeatureEngineer(spec).fit_transform(quotes)
assert result.to_dict(as_series=False) == {"spread": [2.0], "mid": [10.0]}

FeatureEngineer accepts a single FeatureSpec directly, as in the example above, a dict of FeatureSpec keyword arguments, or an ordered sequence mixing those forms. Dicts are unpacked with FeatureSpec(**config) and undergo the same validation. For example:

import polars as pl

from polars_learn import FeatureEngineer

X = pl.DataFrame({"price": [10.0, 20.0]})
engineer = FeatureEngineer({"expression": pl.col("price") * 2, "alias": "doubled"})
assert engineer.fit_transform(X)["doubled"].to_list() == [20.0, 40.0]

A dict uses the exact keys expression, alias, drop, name, and verbosity accepted by FeatureSpec. The required expression value accepts one Polars expression or a nonempty sequence; strings, generators, and empty sequences are rejected. Expressions must be actual Polars expression objects; expression strings are not evaluated. Unknown keys and invalid values fail at construction, with the configuration's sequence index included when applicable. An empty dict is invalid; use FeatureEngineer([]) for an identity operation.

Single definitions are treated as one-element sequences internally; constructor parameters, including original dicts, are retained unchanged for scikit-learn cloning and get_params. The same forms are accepted by set_params(features=...); refit after changes.

Expressions within one specification run together. They can reference columns already present, but cannot reference columns newly created by another expression in that same specification. Use separate specifications for dependent features. Specifications run in list order; drops occur after each specification's expressions. Later specifications cannot use dropped columns.

Existing columns can be overwritten: FeatureSpec(pl.col("price") + 1) replaces price. Selectors such as pl.all().name.suffix("_copy") can expand to multiple outputs. Duplicate output names and missing expression or drop inputs fail during schema checking; duplicates detectable when constructing a specification fail there. An empty FeatureEngineer([]) leaves the columns unchanged.

FeatureEngineer.transform requires the fitted input names in exactly the same order, and the resulting output names, order, and dtypes must match the fitted output schema. Input dtypes may differ if the final output schema still matches. Use a SchemaValidator first when input types need a defined conversion policy.

Fitting eager input evaluates the feature expressions. Fitting lazy input checks the plan's schema without evaluating rows. Features do not learn statistics; use deterministic expressions. Currently fit_transform calls both fit and transform, so eager feature expressions are evaluated twice.

Fixed Contracts and Column Selection

SchemaSpec checks an explicit, nonempty schema. Required columns must have exactly the specified dtypes; this class does not cast them. Both its fit and transform validate input, and transform requires a successful fit.

ColumnSelector returns just the columns you choose, in the requested order. Unselected columns are ignored, input order may differ, and dtypes are not validated. Selected and renamed output names must be unique.

import polars as pl

from polars_learn import ColumnSelector, SchemaSpec

X = pl.DataFrame({"price": [10.0, 20.0], "quantity": [2, 3]})
contract = SchemaSpec({"price": pl.Float64, "quantity": pl.Int64})
validated = contract.fit_transform(X)

selector = ColumnSelector(["quantity", "price"], rename={"quantity": "qty"})
selected = selector.fit_transform(validated)
assert selected.columns == ["qty", "price"]

For SchemaSpec, extra columns are rejected by default. If extras are allowed, extra_columns="drop" produces only the required columns; "preserve" puts required columns first and then extras in their input order. Preserved extra column names become part of the fitted output names. Later transforms must produce those same names in the same order; their dtypes are not checked. Use SchemaValidator if inference may introduce new extra columns.

Explicit Feature Passthrough

Use FeaturePassthrough(columns) when a pipeline layer should simply carry named columns forward. It returns just those columns in the specified order, with their Polars values, names, dtypes, nulls, and row order preserved. There is no renaming or casting option. Selected columns must exist; input order and dtypes may differ between calls. It supports the same verbosity, pandas conversion, fitted feature names, and output controls as the other transformers.

import polars as pl
from sklearn.pipeline import Pipeline

from polars_learn import FeaturePassthrough

X = pl.DataFrame({"id": [7, 8], "flag": [True, False], "unused": [1, 2]})
pipeline = Pipeline([("unchanged", FeaturePassthrough(["id", "flag"]))])
result = pipeline.fit_transform(X.lazy())
assert isinstance(result, pl.LazyFrame)
assert result.collect().equals(X.select("id", "flag"))

A sequential Pipeline passes each step's output to the next step, so a passthrough step selecting two columns omits all the others. To keep unchanged columns alongside transformed columns, put the operations in parallel ColumnTransformer branches:

import polars as pl
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler

from polars_learn import FeaturePassthrough

X = pl.DataFrame({"value": [10.0, 20.0], "id": [7, 8], "flag": [True, False]})
preprocessing = ColumnTransformer(
    [
        ("scaled", StandardScaler(), ["value"]),
        ("unchanged", FeaturePassthrough(["id", "flag"]), ["id", "flag"]),
    ],
    verbose_feature_names_out=False,
).set_output(transform="polars")
result = preprocessing.fit_transform(X)
assert result["value"].to_list() == [-1.0, 1.0]
assert result.select("id", "flag").equals(X.select("id", "flag"))
assert result.columns == ["value", "id", "flag"]

The outer column lists tell ColumnTransformer which input to send to each branch; FeaturePassthrough declares its own selected-column contract. Use eager input at this boundary (lazy_X.collect() for a lazy query). Polars output keeps the separate column dtypes. verbose_feature_names_out=False prevents branch prefixes on output names; final names must then be unique. For branch behavior, see the scikit-learn ColumnTransformer reference. Use ColumnSelector when you also need optional renaming.

Learned Schema Validation

SchemaValidator.fit learns all training columns as required columns. Transform checks them against the learned dtypes according to mode:

Mode	Treatment of a different input dtype
"strict" (default)	Reject it; no casting.
"semi-strict"	Allow only the lossless widening policy below.
"permissive"	Attempt a Polars cast with strict=True; some successful casts may lose precision.

Semi-strict permits integer widening when the entire source dtype's range fits inside the fitted integer dtype's range, including compatible unsigned to signed widening. Small observed values alone do not make narrowing acceptable. It also permits Float32 to Float64, and integer to Float64 only when every non-null value round-trips exactly. For example, 2**53 and 2**54 are accepted, but 2**53 + 1 is rejected. String to numeric conversion, narrowing, and Null dtype to another dtype are rejected in this mode.

import polars as pl

from polars_learn import SchemaValidator

train = pl.DataFrame({"x": pl.Series([1], dtype=pl.Int64)})
incoming = pl.DataFrame({"x": pl.Series([2, None], dtype=pl.Int32)})
validator = SchemaValidator(mode="semi-strict").fit(train)
result = validator.transform(incoming)
assert result.schema["x"] == pl.Int64
assert result["x"].to_list() == [2, None]

Required columns must always be present. Extra columns fail unless allow_extra_columns=True; allowing extras does not make fitted columns optional. extra_columns="drop" removes allowed extras, while the default "preserve" keeps them. The drop setting alone does not allow extras.

enforce_column_order=True checks the relative order of required columns, so allowed extras can appear between them. With enforce_column_order=False, output puts fitted columns first, followed by preserved extras in their input order. get_feature_names_out() lists required fitted columns, excluding inference-only extras. Preserving extras can change the output width; drop them before an estimator that expects a fixed number of features.

Null values are preserved when the dtype is compatible. Nullability constraints are not learned. In permissive mode, a column with Null dtype can be cast to the fitted dtype. Failed strict casts raise; they do not silently replace invalid values with nulls. Choose the validation policy through mode: "strict", "semi-strict", or "permissive". The default is "strict".

LazyFrames and Return Types

All five transformers accept a pl.LazyFrame, including a query from pl.scan_csv(...). Default output follows the input to each call, independently of whether fitting used eager or lazy input. Pandas input, when conversion is allowed, is treated as an eager DataFrame.

set_output(transform=...)	DataFrame input	LazyFrame input
"default" (initial behavior)	DataFrame	LazyFrame
"polars"	DataFrame	DataFrame; collects the result
"polars-lazy"	LazyFrame	LazyFrame
None	Keep the existing choice	Keep the existing choice

import polars as pl

from polars_learn import FeatureEngineer, FeatureSpec

X = pl.DataFrame({"price": [10.0, 20.0], "quantity": [2, 3]})
engineer = FeatureEngineer(
    [FeatureSpec(pl.col("price") * pl.col("quantity"), alias="notional")]
)
plan = engineer.fit_transform(X.lazy())  # Build a plan; do not evaluate rows
assert isinstance(plan, pl.LazyFrame)
result = plan.collect()  # Execute when ready
assert result["notional"].to_list() == [20.0, 60.0]

engineer.set_output(transform="polars")
eager_result = engineer.transform(X.lazy())  # Collect inside transform
assert isinstance(eager_result, pl.DataFrame)

engineer.set_output(transform="polars-lazy")
lazy_result = engineer.transform(X)
assert isinstance(lazy_result, pl.LazyFrame)
engineer.set_output(transform="default")

Output settings control transform and fit_transform returns. fit always returns the transformer and does not collect lazy rows just because eager output is configured. Forcing a lazy return from eager input can wrap an already computed result; pass lazy input to keep row evaluation deferred throughout.

Lazy operations check column names, order, dtypes, expression dependencies, and drops using schema metadata. Resolving metadata can access a source's metadata; it does not collect all its rows. Lazy fit validates the plan schema, not every input value. Value-dependent errors surface at collection. A returned LazyFrame is a query plan: collecting it again may execute the query again.

Inspect lazy output with plan.collect_schema() for names and dtypes, and plan.collect() for data. Verbosity reports lazy row counts as unknown rather than evaluating rows to count them.

Pipelines with scikit-learn-style Estimators

The preprocessing steps use fit and transform; the final estimator uses fit(X, y) and its prediction method. Pass raw data to the complete pipeline for both fitting and prediction. Pipeline fits preprocessing on training input and, during predict, reapplies the fitted schema policies and feature expressions before invoking the model. The transformers ignore y; the model uses it to learn predictions.

A pipeline built from these transformers can remain lazy. Set the last native transformer to eager output before a downstream estimator that needs materialized data. "polars-lazy" is a package-specific output option; do not pass it to scikit-learn's standard transformers.

import numpy as np
import polars as pl
from sklearn.ensemble import RandomForestRegressor
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

from polars_learn import FeatureEngineer, FeatureSpec, SchemaValidator

X = pl.DataFrame({"price": [10.0, 20.0, 30.0], "quantity": [2, 3, 4]})
y = np.array([1.0, 2.0, 3.0])
pipeline = Pipeline(
    [
        ("schema", SchemaValidator(mode="semi-strict")),
        (
            "features",
            FeatureEngineer(
                [FeatureSpec(pl.col("price") * pl.col("quantity"), alias="notional")]
            ).set_output(transform="polars"),
        ),
        ("scale", StandardScaler().set_output(transform="polars")),
        ("model", RandomForestRegressor(n_estimators=20, random_state=0)),
    ]
)
pipeline.fit(X.lazy(), y)
predictions = pipeline.predict(X.lazy())
assert predictions.shape == (3,)

Supported input is Polars DataFrames and LazyFrames, plus explicitly enabled pandas DataFrames. NumPy arrays, Series, Python lists, and sparse arrays are not accepted. These transformers bypass scikit-learn's automatic output wrapping and global output-container settings; configure each native transformer with set_output. There is no automatic conversion to NumPy or pandas output.

Cloning keeps constructor parameters and explicit output settings, and discards fitted state. y is accepted for estimator compatibility and ignored. Additional fit_transform keyword parameters, including sample_weight, are unsupported. Refit after changing parameters that affect the fitted contract; set_params does not reset fitted state. Invalid settings are rechecked when processing input.

The Estimator Pattern

These transformers follow the same estimator conventions used by scikit-learn and libraries such as XGBoost:

Operation	Preprocessing transformer	Final predictive model
fit	Validate or learn the training input contract; return the fitted transformer.	Learn model parameters from training features and targets.
Apply fitted state	transform(X) returns prepared features.	predict(X) returns predictions.
Fit and apply	fit_transform(X) fits and returns prepared training features.	Fit first, then call the model's prediction method.
Configure	Constructor parameters, get_params, and set_params.	The same parameter conventions for estimator-style models.

In a Pipeline, each preprocessing step prepares the input for the next step, and the final estimator learns from that prepared input. Calling predict on the fitted pipeline reapplies the preprocessing steps before predicting. XGBoost is an example of a library with a scikit-learn estimator interface; its regression/classification models use fit and predict. The transform operation belongs to the preprocessing steps.

This shared interface describes how steps compose. The final estimator must also accept the prepared feature container and dtypes. Set the last native transformer to set_output(transform="polars") when the model needs eager data. If a model requires a different container, add an explicit conversion step at that boundary. Keep preprocessing fitted on training input and reuse it for validation and prediction.

Saving and Loading a Fitted Pipeline

Save the complete fitted pipeline so the feature expressions, raw-input contract, fitted feature names, output settings, and model stay together. After loading it, keep passing raw observations to predict; you do not need to recreate the Polars expressions or prepare features separately.

FeatureSpec, FeatureEngineer, ColumnSelector, FeaturePassthrough, SchemaSpec, and SchemaValidator support Python pickling in fitted and unfitted forms where applicable. Ordinary pl.Expr objects, single-spec inputs, dict configs, and mixed specification sequences are preserved using Polars' native expression serialization. Fitted schemas and output-container choices survive loading. Pickling an unfitted transformer does not fit it.

import pickle
from pathlib import Path
from tempfile import TemporaryDirectory

import numpy as np
import polars as pl
from sklearn.ensemble import RandomForestRegressor
from sklearn.pipeline import Pipeline

from polars_learn import FeatureEngineer, FeatureSpec, SchemaValidator

raw_train = pl.DataFrame({"price": [10.0, 20.0, 30.0], "quantity": [2, 3, 4]})
y = np.array([1.0, 2.0, 3.0])
pipeline = Pipeline(
    [
        ("schema", SchemaValidator()),
        (
            "features",
            FeatureEngineer(
                FeatureSpec(pl.col("price") * pl.col("quantity"), alias="notional")
            ).set_output(transform="polars"),
        ),
        ("model", RandomForestRegressor(n_estimators=10, random_state=0)),
    ]
)
pipeline.fit(raw_train, y)

# Use your persistent model path in an application; this example cleans up its files.
with TemporaryDirectory() as directory:
    model_path = Path(directory) / "pipeline.pkl"
    with model_path.open("wb") as destination:
        pickle.dump(pipeline, destination, protocol=pickle.HIGHEST_PROTOCOL)
    with model_path.open("rb") as source:
        restored = pickle.load(source)

raw_predict = pl.DataFrame({"price": [15.0, 25.0], "quantity": [2, 4]})
np.testing.assert_array_equal(
    restored.predict(raw_predict), pipeline.predict(raw_predict)
)
assert restored[:-1].get_feature_names_out().tolist() == [
    "price",
    "quantity",
    "notional",
]

joblib.dump/joblib.load and cloudpickle.dumps/cloudpickle.loads also work with these objects. cloudpickle is a runtime dependency because Python callbacks embedded in Polars expressions need it for serialization. Tests include local lambda/closure callbacks and loading a complete fitted pipeline in a fresh interpreter. Prefer native Polars expressions when possible; custom callbacks must have serializable captured state and their required modules, plugins, and external resources available when they execute.

Saved objects record their creating versions of Python, polars-learn, Polars, scikit-learn, NumPy, and cloudpickle. Loading under a different version emits polars_learn.InconsistentVersionWarning for each changed dependency before restoring that object's state, then attempts loading. The warning exposes class_name, dependency, original_version, and current_version. Older artifacts without version metadata cannot report their creating environment.

Same-version expression loading uses Polars binary serialization to preserve native values, including NaN and infinity. Cross-version expression loading tries Polars' JSON representation, provided the expression operations, dtypes, callbacks, and other required objects are available and compatible in the receiving environment. If that representation is unavailable or rejected, loading raises SerializationError with version context and the original exception as its cause; it does not fall back to the old binary layout. CI verifies both directions between Polars 1.35.2 and the locked version, currently 1.44.2, for all six main classes and a fitted prediction pipeline.

To reject a version mismatch before restoring state, promote the warning to an error:

import pickle
import warnings

import polars as pl

from polars_learn import FeatureSpec, InconsistentVersionWarning

saved = pickle.dumps(FeatureSpec(pl.col("price") * 2))
with warnings.catch_warnings():
    warnings.simplefilter("error", InconsistentVersionWarning)
    restored_spec = pickle.loads(saved)

If you accept a version mismatch, silence this package's version warnings around loading with the standard Python warning filter. The context restores your previous filters afterward and leaves other warning categories active. This works with pickle, joblib, and cloudpickle; incompatible state still raises SerializationError.

import pickle
import warnings

import polars as pl

from polars_learn import FeatureSpec, InconsistentVersionWarning

saved = pickle.dumps(FeatureSpec(pl.col("price") * 2))
with warnings.catch_warnings():
    warnings.simplefilter("ignore", InconsistentVersionWarning)
    restored_spec = pickle.loads(saved)

To silence this category application-wide, call warnings.filterwarnings("ignore", category=InconsistentVersionWarning) once in your application. Final estimators may issue their own version warnings, controlled by their own warning categories.

Load only artifacts you trust. Pickle-based loading can execute Python code, including through serialized expressions containing Python callbacks. Cross-version loading is a compatibility attempt, and a successful load does not guarantee identical numerical results. Validate transformed features and predictions after upgrading, and retain the original environment lockfile for recovery. Arbitrary future version transitions and unpicklable captured resources cannot be guaranteed portable. See Polars expression persistence and scikit-learn model persistence.

Verbosity

All six classes accept an integer verbosity:

Level	Output to standard output
0 (default)	No progress diagnostics.
1	Input dimensions and fit/transform progress; specification construction summaries.
2	Level 1 plus schemas and feature operation details.

import polars as pl

from polars_learn import SchemaValidator

X = pl.DataFrame({"x": [1, 2]})
validator = SchemaValidator(verbosity=1)
validated = validator.fit_transform(X)
# [SchemaValidator] Validating input
# [SchemaValidator] Input: 2 rows, 1 columns
# [SchemaValidator] Fit complete; output columns: ['x']
# [SchemaValidator] Validating input
# [SchemaValidator] Input: 2 rows, 1 columns
# [SchemaValidator] Transform complete: 2 rows, 1 columns

FeatureSpec reports its construction separately. An engineer's verbosity does not change its specifications' verbosity. A dict config's verbosity applies when that config is converted into a specification, during construction and each fit/transform call. Frame summaries show metadata rather than printing data rows; feature details can include expression representations. Levels must be 0, 1, or 2; booleans and strings are rejected. Verbosity does not suppress exceptions or pandas conversion warnings.

Errors and When They Occur

Situation	Exception or warning	Timing
Invalid argument type or unsupported input	TypeError	Construction or the next relevant operation
Invalid settings, conflicting policies, or duplicate configured names	ValueError	Construction or revalidation
Transform or feature-name access before fitting	sklearn.exceptions.NotFittedError	Method call
SchemaSpec contract or fitted output-name mismatch	ValueError	Fit/transform schema check
ColumnSelector/FeaturePassthrough missing columns or fitted output-name mismatch	ValueError	Fit/transform schema check
SchemaValidator missing columns, disallowed extras/order/types, or eager cast failure	SchemaValidationError	Transform
Feature dependency, drop, or schema failure; eager feature execution failure	FeatureEngineeringError	Fit/transform
Value-dependent feature failure or permissive cast failure in a returned lazy plan	Native Polars exception	collect() or another execution operation
Lossy semi-strict integer to Float64 cast in a lazy plan	SchemaValidationError	Lazy execution
Enabled pandas conversion with warnings enabled	PandasConversionWarning	Conversion, even at verbosity 0

PolarsLearnError is the base of SchemaValidationError and FeatureEngineeringError. Plain TypeError/ValueError, NotFittedError, and native Polars errors do not inherit from it. Input query and source failures can also propagate as native Polars or source exceptions during schema resolution or collection. Wrapped errors preserve their original exception as __cause__.

To force value evaluation within transform, request "polars" output. FeatureEngineer wraps resulting execution errors in FeatureEngineeringError; SchemaValidator wraps Polars collection errors in SchemaValidationError. SchemaSpec, ColumnSelector, and FeaturePassthrough do not wrap source/collection errors.

Optional Pandas Support

Install the pandas extra shown above, then explicitly allow conversion:

import pandas as pd

from polars_learn import SchemaValidator

X = pd.DataFrame({"price": [10.0, 20.0], "quantity": [2, 3]})
validator = SchemaValidator(convert_pandas=True, warn_on_conversion=False)
result = validator.fit_transform(X)
assert result.columns == ["price", "quantity"]

All five transformers expose convert_pandas and warn_on_conversion. Conversion is disabled by default. When enabled, it emits PandasConversionWarning unless warn_on_conversion=False. Use Polars directly to avoid conversion overhead; fit_transform may convert input more than once. Pandas and PyArrow are imported only when needed. The extra includes PyArrow for nullable, extension, and other dtypes that require it. Conversion follows Polars conventions, including NaN to null conversion and ignoring the pandas index. Unsupported dtypes raise; conversion does not fall back to row operations. An explicit "polars-lazy" output choice returns a lazy wrapper after conversion.

Development

Run from a repository checkout:

uv sync  # Includes pandas and PyArrow for the full test suite
uv run pytest --doctest-modules src tests
uv run ruff check .
uv run ruff format .
uv run ruff format --check .
uv run pyright
uv build

Ruff is the formatter for all Python code and documentation examples. CI checks ruff format --check ., linting, types, tests, and API doctests. Ruff's directory checks cover Python files, not Markdown code fences; format a Markdown example with ruff format --stdin-filename example.py before adding it. The test matrix covers Python 3.11 and 3.12 with all development dependencies, including pandas and PyArrow. Every job runs the full interoperability suite. A subprocess test separately verifies native usage with pandas and PyArrow blocked.

uv.lock pins development dependencies. Pyright includes its Node runtime via the development group. Pandas and PyArrow are required for development; missing pandas causes test collection to fail rather than silently skip coverage. Run uv sync after changing requirements. These remain optional runtime dependencies for users of the package. The package includes py.typed; transformer return annotations cover both DataFrame and LazyFrame because set_output can change the container. Narrow with isinstance(result, pl.LazyFrame) when your type checker needs the specific type. Library dependencies use minimum versions; the lockfile does not constrain downstream applications to those exact versions.

Releases

Run Actions → Create GitHub Release → Run workflow to release the version in pyproject.toml from the selected branch or tag. The workflow validates the package, builds a fresh wheel and source archive, and creates a GitHub release named vX.Y.Z with those files attached. Once that workflow succeeds, Upload Python Package starts automatically and publishes the same built files to PyPI. Existing files in a local dist/ directory are not used.

Before the first release:

1. Push both release and publishing workflows to the repository's default branch. The automatic publisher uses GitHub's workflow_run event, which requires the publishing workflow on that branch.
2. Create the GitHub Actions environment pypi if it does not already exist.
3. Configure the polars-learn PyPI project's trusted publisher with your GitHub repository owner/name, workflow filename python-publish.yml, and environment pypi. See PyPI's setup instructions. Publishing uses OIDC and does not read a PYPI_API_TOKEN secret.

For each new version:

1. Set project.version in pyproject.toml and __version__ in src/polars_learn/__init__.py to the same new X.Y.Z version, then run uv lock.
2. Commit and push the release changes. The runner builds from the selected committed revision, so uncommitted local changes are not included.
3. Run Create GitHub Release from Actions, selecting the release branch or tag.
4. Check both workflow runs. The second run uses the pypi environment; any protection rules configured for that environment still apply.

The release workflow fails on mismatched versions, an existing vX.Y.Z tag, failed tests/checks, or invalid build metadata. The automatic publisher only accepts successful manual release-workflow runs from this repository and downloads artifacts from that exact run. Failed or cancelled release runs do not publish. It works with GITHUB_TOKEN without relying on its release event to start another workflow. See GitHub's workflow_run documentation.

Publishing a GitHub release manually through the Releases page is also supported. Use a vX.Y.Z tag matching the code's package version. That path builds and validates the tagged source before uploading to PyPI.

If PyPI publishing fails after the GitHub release was created, fix the reported setup or upload problem and rerun Upload Python Package, rather than creating the same release tag again. Its automatic path reuses the original release-run artifacts, which must still be available. PyPI versions cannot be overwritten; use a new version for changed distributions.

License

MIT. See LICENSE.