Polypolars

Generate type-safe Polars DataFrames effortlessly using polyfactory

Inspired by polyspark, polypolars lets you create realistic test DataFrames from your Python data models—with automatic schema inference for Polars.

Docs: See the docs/ folder and run mkdocs serve for the full API reference and examples.

Example

from dataclasses import dataclass
from polypolars import polars_factory

@polars_factory
@dataclass
class User:
    id: int
    name: str
    email: str

# Generate 1000 rows instantly:
df = User.build_dataframe(size=1000)
print(df.head())

Example output (data varies per run):

shape: (5, 3)
┌──────┬──────────────────────┬──────────────────────┐
│ id   ┆ name                 ┆ email                │
│ ---  ┆ ---                  ┆ ---                  │
│ i64  ┆ str                  ┆ str                  │
╞══════╪══════════════════════╪══════════════════════╡
│ 3167 ┆ QmYHeLMDMxWChjihAFxU ┆ vHGMKHjXsMBlxLuhqpUE │
│ 1028 ┆ hvLXPtlqURtwzqeyJruo ┆ ePDAdtelIEiRfEuAgoPz │
│ 9048 ┆ NhnyGGQsTjxPEndxaOCt ┆ znmByWtpwofUGKolkJrs │
│  971 ┆ ZlkxcjcVAZfLUkCwHRFG ┆ PTtzmMHcvLQPcOrAgFpl │
│ 3813 ┆ tIqqrgyYjULzdyRKkMKK ┆ tMAFeQewaQFtRGEvOdqW │
└──────┴──────────────────────┴──────────────────────┘

Contents

• Why Polypolars? · Installation · Quick Start · Schema inference · Type mapping · CLI · I/O and testing

Why Polypolars?

• Factory pattern: Leverage polyfactory for data generation
• Type-safe schema: Python types become Polars dtypes automatically
• Nullable handling: Optional[T] and defaults are reflected in the schema
• Complex types: Nested structs, lists, and dicts (as list-of-structs)
• Multiple models: Dataclasses, Pydantic, and TypedDict

Installation

pip install polypolars

For development:

pip install "polypolars[dev]"

Quick Start

Decorator (recommended)

from dataclasses import dataclass
from typing import Optional
from polypolars import polars_factory

@polars_factory
@dataclass
class Product:
    product_id: int
    name: str
    price: float
    description: Optional[str] = None
    in_stock: bool = True

# Build Polars DataFrame
df = Product.build_dataframe(size=100)
print(df.head())

# Or get dicts
dicts = Product.build_dicts(size=50)

Example output (first 5 rows; data varies per run):

shape: (5, 5)
┌────────────┬──────────────────────┬──────────────┬──────────────────────┬──────────┐
│ product_id ┆ name                 ┆ price        ┆ description          ┆ in_stock │
│ ---        ┆ ---                  ┆ ---          ┆ ---                  ┆ ---      │
│ i64        ┆ str                  ┆ f64          ┆ str                  ┆ bool     │
╞════════════╪══════════════════════╪══════════════╪══════════════════════╪══════════╡
│ 5582       ┆ hKJsoOOXlwgLIiiWOCJP ┆ 2.2760e8     ┆ rTUACBLlGBlHXIjzVvPt ┆ false    │
│ 7099       ┆ ZgUiDVJirxAYRrWIPnpS ┆ 274887.17671 ┆ bHGMXNFRLSDifpywMZrY ┆ true     │
│ 5372       ┆ MTtVHJkqneaCkoyZNgio ┆ 1.5195e7     ┆ HsAmRwgaphvQxOCJwjSr ┆ false    │
│ 8650       ┆ fTBYFPiWMFCKauieEXlu ┆ -7.8765e8    ┆ UAnyfVhTUmvcjtzbCufq ┆ true     │
│ 1023       ┆ MCtTOwvJTjfbpPELcFKm ┆ -97.933431   ┆ PMEHaEOGaoJiDaomXdVX ┆ false    │
└────────────┴──────────────────────┴──────────────┴──────────────────────┴──────────┘

Classic factory class

from polypolars import PolarsFactory

class ProductFactory(PolarsFactory[Product]):
    __model__ = Product

df = ProductFactory.build_dataframe(size=100)

Convenience function

from polypolars import build_polars_dataframe

df = build_polars_dataframe(Product, size=100)

Schema inference

Schema is inferred from your type hints, so all-null columns still get the correct type:

@polars_factory
@dataclass
class User:
    id: int
    email: Optional[str]  # nullable string in Polars

df = User.build_dataframe(size=100)  # schema: id Int64, email String

From dicts

dicts = Product.build_dicts(size=1000)
# Convert to DataFrame when needed:
df = Product.create_dataframe_from_dicts(dicts)

Pydantic

from pydantic import BaseModel, Field
from polypolars import polars_factory

@polars_factory
class User(BaseModel):
    id: int = Field(gt=0)
    username: str = Field(min_length=3, max_length=20)
    email: str
    is_active: bool = True

df = User.build_dataframe(size=500)

Type mapping

Use schema_overrides (e.g. {"col": pl.Categorical}) to override inferred types.

LazyFrame and chunked building

# LazyFrame
lf = Product.build_lazy_dataframe(size=10_000)

# Chunked building for very large size (lower memory)
df = Product.build_dataframe(size=1_000_000, chunk_size=10_000)

CLI

# Export schema
polypolars schema export myapp.models:User --output schema.txt

# Validate a file against a model
polypolars schema validate myapp.models:User data.parquet

# Generate sample data
polypolars generate myapp.models:User --size 1000 --output users.parquet --format parquet

I/O and testing

from polypolars import (
    save_as_parquet,
    load_parquet,
    load_and_validate,
    infer_schema,
    assert_dataframe_equal,
    assert_schema_equal,
)

df = User.build_dataframe(size=1000)
save_as_parquet(df, "users.parquet")

# Load and validate
schema = infer_schema(User)
df2 = load_and_validate("users.parquet", expected_schema=schema)

assert_dataframe_equal(df, df2, check_order=False)

License

MIT

Related

• polyspark – inspiration for this library
• polyfactory – factory library for mock data
• Polars – fast DataFrame library