pydantable 1.18.0

Strongly-typed DataFrames for Python, powered by Rust.

PydanTable

Strongly typed DataFrames for Python, powered by Rust.

PydanTable combines Pydantic schemas with a Polars-backed Rust execution engine to provide a typed, service-friendly DataFrame API (with optional integrations for FastAPI, SQL, MongoDB, Spark, and more).

Current release: 1.18.0 — highlights in the changelog.

Documentation

• Docs (latest): pydantable.readthedocs.io
• Quickstart: Getting started → Quickstart
• Docs map: Getting started → Docs map

What you get

• Typed tables via Pydantic models: DataFrameModel or DataFrame[Schema]
• Typed expressions + lazy plans validated/lowered in Rust
• Explicit materialization: collect() (rows) or to_dict() (columns), plus optional Arrow/Polars exports
• File / HTTP / SQL I/O helpers and integration patterns for services

Key references:

• DataFrameModel: User guide → DataFrameModel
• Execution: User guide → Execution
• Materialization: User guide → Materialization
• Interface contract: Semantics → Interface contract
• I/O overview: I/O → Overview

Install

pip install pydantable

Optional extras:

pip install "pydantable[polars]"   # to_polars
pip install "pydantable[arrow]"    # to_arrow / Arrow constructors
pip install "pydantable[io]"       # full file I/O convenience (arrow + polars)
pip install "pydantable[sql]"      # SQLModel + SQLAlchemy + moltres-core lazy SqlDataFrame; add a DB-API driver for your URL
pip install "pydantable[pandas]"   # pandas-flavored façade (pandas UI doc)
pip install "pydantable[fastapi]"  # FastAPI integration (pydantable.fastapi)
pip install "pydantable[mongo]"    # pymongo + Beanie + Mongo plan stack (lazy MongoDataFrame + I/O + from_beanie)
pip install "pydantable[spark]"    # SparkDataFrame / SparkDataFrameModel (raikou-core + pyspark + sparkdantic)

Quick start

from pydantable import DataFrameModel

class User(DataFrameModel):
    id: int
    age: int | None

df = User({"id": [1, 2], "age": [20, None]})
result = (
    df.with_columns(age2=df.age * 2)
    .filter(df.age > 10)
    .select("id", "age2")
)

print(result.to_dict())
print([r.model_dump() for r in result.collect()])

Output (one run):

{'id': [1], 'age2': [40]}
[{'id': 1, 'age2': 40}]

Next steps

• Start here: Quickstart
• Typing guide: User guide → Typing
• I/O decision tree: I/O → Decision tree
• FastAPI golden path: Integrations → FastAPI → Golden path
• Engines: SQL · Mongo · Spark

Development

Use a virtual environment at .venv in the repo root (the Makefile defaults to .venv/bin/python). Full contributor setup, native builds, and contributor notes: Project → Developer.

make check-full      # ruff, ty, pyright, typing snippet tests, MkDocs, Rust

License

MIT