Polypolars
Generate type-safe Polars DataFrames effortlessly using polyfactory
Inspired by polyspark, polypolars lets you create realistic test DataFrames from your Python data models—with automatic schema inference for Polars.
Docs: See the docs/ folder and run mkdocs serve for the full API reference and examples.
Example
from dataclasses import dataclass
from polypolars import polars_factory
@polars_factory
@dataclass
class User:
id: int
name: str
email: str
# Generate 1000 rows instantly:
df = User.build_dataframe(size=1000)
print(df.head())
Example output (data varies per run):
shape: (5, 3)
┌──────┬──────────────────────┬──────────────────────┐
│ id ┆ name ┆ email │
│ --- ┆ --- ┆ --- │
│ i64 ┆ str ┆ str │
╞══════╪══════════════════════╪══════════════════════╡
│ 3167 ┆ QmYHeLMDMxWChjihAFxU ┆ vHGMKHjXsMBlxLuhqpUE │
│ 1028 ┆ hvLXPtlqURtwzqeyJruo ┆ ePDAdtelIEiRfEuAgoPz │
│ 9048 ┆ NhnyGGQsTjxPEndxaOCt ┆ znmByWtpwofUGKolkJrs │
│ 971 ┆ ZlkxcjcVAZfLUkCwHRFG ┆ PTtzmMHcvLQPcOrAgFpl │
│ 3813 ┆ tIqqrgyYjULzdyRKkMKK ┆ tMAFeQewaQFtRGEvOdqW │
└──────┴──────────────────────┴──────────────────────┘
Contents
- Why Polypolars? · Installation · Quick Start · Schema inference · Type mapping · CLI · I/O and testing
Why Polypolars?
- Factory pattern: Leverage polyfactory for data generation
- Type-safe schema: Python types become Polars dtypes automatically
- Nullable handling:
Optional[T]and defaults are reflected in the schema - Complex types: Nested structs, lists, and dicts (as list-of-structs)
- Multiple models: Dataclasses, Pydantic, and TypedDict
Installation
pip install polypolars
For development:
pip install "polypolars[dev]"
Quick Start
Decorator (recommended)
from dataclasses import dataclass
from typing import Optional
from polypolars import polars_factory
@polars_factory
@dataclass
class Product:
product_id: int
name: str
price: float
description: Optional[str] = None
in_stock: bool = True
# Build Polars DataFrame
df = Product.build_dataframe(size=100)
print(df.head())
# Or get dicts
dicts = Product.build_dicts(size=50)
Example output (first 5 rows; data varies per run):
shape: (5, 5)
┌────────────┬──────────────────────┬──────────────┬──────────────────────┬──────────┐
│ product_id ┆ name ┆ price ┆ description ┆ in_stock │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ i64 ┆ str ┆ f64 ┆ str ┆ bool │
╞════════════╪══════════════════════╪══════════════╪══════════════════════╪══════════╡
│ 5582 ┆ hKJsoOOXlwgLIiiWOCJP ┆ 2.2760e8 ┆ rTUACBLlGBlHXIjzVvPt ┆ false │
│ 7099 ┆ ZgUiDVJirxAYRrWIPnpS ┆ 274887.17671 ┆ bHGMXNFRLSDifpywMZrY ┆ true │
│ 5372 ┆ MTtVHJkqneaCkoyZNgio ┆ 1.5195e7 ┆ HsAmRwgaphvQxOCJwjSr ┆ false │
│ 8650 ┆ fTBYFPiWMFCKauieEXlu ┆ -7.8765e8 ┆ UAnyfVhTUmvcjtzbCufq ┆ true │
│ 1023 ┆ MCtTOwvJTjfbpPELcFKm ┆ -97.933431 ┆ PMEHaEOGaoJiDaomXdVX ┆ false │
└────────────┴──────────────────────┴──────────────┴──────────────────────┴──────────┘
Classic factory class
from polypolars import PolarsFactory
class ProductFactory(PolarsFactory[Product]):
__model__ = Product
df = ProductFactory.build_dataframe(size=100)
Convenience function
from polypolars import build_polars_dataframe
df = build_polars_dataframe(Product, size=100)
Schema inference
Schema is inferred from your type hints, so all-null columns still get the correct type:
@polars_factory
@dataclass
class User:
id: int
email: Optional[str] # nullable string in Polars
df = User.build_dataframe(size=100) # schema: id Int64, email String
From dicts
dicts = Product.build_dicts(size=1000)
# Convert to DataFrame when needed:
df = Product.create_dataframe_from_dicts(dicts)
Pydantic
from pydantic import BaseModel, Field
from polypolars import polars_factory
@polars_factory
class User(BaseModel):
id: int = Field(gt=0)
username: str = Field(min_length=3, max_length=20)
email: str
is_active: bool = True
df = User.build_dataframe(size=500)
Type mapping
| Python | Polars |
|---|---|
str |
String |
int |
Int64 |
float |
Float64 |
bool |
Boolean |
datetime |
Datetime |
date |
Date |
List[T] |
List(T) |
Dict[K,V] |
List(Struct(key, value)) |
Optional[T] |
T (nullable) |
Tuple[T, ...] |
List(T) |
Tuple[T, T, ...] (fixed) |
Array(T, n) |
| Dataclass / Pydantic | Struct(...) |
Use schema_overrides (e.g. {"col": pl.Categorical}) to override inferred types.
LazyFrame and chunked building
# LazyFrame
lf = Product.build_lazy_dataframe(size=10_000)
# Chunked building for very large size (lower memory)
df = Product.build_dataframe(size=1_000_000, chunk_size=10_000)
CLI
# Export schema
polypolars schema export myapp.models:User --output schema.txt
# Validate a file against a model
polypolars schema validate myapp.models:User data.parquet
# Generate sample data
polypolars generate myapp.models:User --size 1000 --output users.parquet --format parquet
I/O and testing
from polypolars import (
save_as_parquet,
load_parquet,
load_and_validate,
infer_schema,
assert_dataframe_equal,
assert_schema_equal,
)
df = User.build_dataframe(size=1000)
save_as_parquet(df, "users.parquet")
# Load and validate
schema = infer_schema(User)
df2 = load_and_validate("users.parquet", expected_schema=schema)
assert_dataframe_equal(df, df2, check_order=False)
License
MIT
Related
- polyspark – inspiration for this library
- polyfactory – factory library for mock data
- Polars – fast DataFrame library
Metadata
Release files for polypolars 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| polypolars-0.1.1.tar.gz | 5.3 kB | Without provenance | Details for polypolars-0.1.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| polypolars-0.1.1-py3-none-any.whl | Python 3 | none | any | Details for polypolars-0.1.1-py3-none-any.whl |
Total release size: 10.4 kB
Release files / polypolars-0.1.1.tar.gz
| Download URL | polypolars-0.1.1.tar.gz |
|---|---|
| Size | 5.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
aefeacd71f503f426980379dac4e90153f6091dc37c1cf8c94cb63cc1da6f0a2
|
|
BLAKE2b-256 checksum How to use checksums |
f19ed0d9432d0c700b80eca0dc1283239496ecf614ec70826b2cf4f520f5d66d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.8.18
|
Release files / polypolars-0.1.1-py3-none-any.whl
| Download URL | polypolars-0.1.1-py3-none-any.whl |
|---|---|
| Size | 5.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
cd1ee593505dca1b672c8cbdbb5673b2f6e36d3dbe165e4b121ece661804a8de
|
|
BLAKE2b-256 checksum How to use checksums |
a785aa8a8d10e244b96f8fb3665afea450be1507a7cc938ad8da37ea28ef43d2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.8.18
|
Security for polypolars 0.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.