pydantable 0.5.0

Strongly-typed DataFrames for Python, powered by Rust.

PydanTable

Typed dataframe transformations for FastAPI and Pydantic services, backed by a Rust execution core.

Current release: 0.5.0 · Python 3.10+

---

Documentation

The full manual lives on Read the Docs:

https://pydantable.readthedocs.io/en/latest/

That site is the supported entry point for concepts, contracts, API notes, and examples. The sections below point to the same pages so you can jump straight from GitHub.

Topic	Read the Docs
Home / overview	Documentation home
DataFrameModel contract (inputs, transforms, collisions, materialization)	DataFrameModel (SQLModel-like)
Column types (scalars, structs, list[T], nullability, unsupported cases)	Supported data types
FastAPI (routers, bodies, collect, responses)	FastAPI integration
Execution model (collect, to_dict, to_polars, optional Python Polars, UI modules)	Execution (Rust engine)
Semantics (nulls, joins, ordering, reshaping, windows — Polars-style contract)	Interface contract
Roadmap (what shipped in 0.5.0, path to v1.0.0)	Roadmap
Why not use Polars directly?	Why not just use Polars?
Pandas-style imports (pydantable.pandas)	Pandas UI
PySpark-style imports (pydantable.pyspark)	PySpark UI
PySpark helpers & parity	PySpark interface · PySpark API parity
Polars parity (scorecard, workflows, transformation roadmap)	Parity scorecard · Polars-style workflows · Transformation parity roadmap
Contributors (build, test, benchmarks, releases)	Developer guide
Plan / vision (architecture phasing)	Plan document
Python API reference (autodoc)	API reference

For copy-paste convenience, the site base URL is:

https://pydantable.readthedocs.io/en/latest/

---

What PydanTable does

PydanTable keeps Pydantic models as the source of truth for:

• column types and nullability (Optional[T] / T | None)
• typed expressions — invalid combinations fail when the expression is built (Rust AST), not only at runtime
• schema evolution — chained transforms produce new model types with stable rules (e.g. with_columns name collisions)

The default API feels Polars-like; optional pydantable.pandas and pydantable.pyspark modules only change naming and imports — execution is always the native core. Details: Execution, Interface contract.

---

Install

pip install pydantable

Optional Python Polars (for to_polars() only):

pip install 'pydantable[polars]'

From a git checkout, the Rust extension must be built (e.g. with maturin):

pip install .

See Developer guide — local setup for maturin develop, release builds, and CI parity.

---

Quick start

from pydantable import DataFrameModel

class User(DataFrameModel):
    id: int
    age: int | None

df = User({"id": [1, 2], "age": [20, None]})
df2 = df.with_columns(age2=df.age * 2)
df3 = df2.select("id", "age2")
df4 = df3.filter(df3.age2 > 10)

print(df4.to_dict())

Example output:

{'age2': [40], 'id': [1]}

• Materialization: collect() returns a list of Pydantic row models; to_dict() returns columnar dict[str, list].

• Alternate UIs:

  from pydantable.pandas import DataFrameModel as PandasDataFrameModel
  from pydantable.pyspark import DataFrameModel as PySparkDataFrameModel
  from pydantable import DataFrameModel as DefaultDataFrameModel
  

More examples: FastAPI integration, Polars-style workflows.

---

Development

ruff format . && ruff check .
pytest -q

Rust + Python: see Developer guide (formatting, maturin, benchmarks, contribution workflow).

---

License

MIT