DataForge

The fastest fake data generator for Python. Zero dependencies. 18M items/second. Drop-in Faker replacement.

pip install dataforge-py

from dataforge import DataForge

forge = DataForge(seed=42)

forge.person.full_name()                  # "James Smith"
forge.internet.email()                    # "james.smith@gmail.com"
forge.person.first_name(count=1_000_000)  # 1M names in ~55ms

Why DataForge?

DataForge isn't a wrapper around Faker — it's a ground-up rewrite with vectorized batch paths, pre-resolved field lookups, lazy locale loading, and columnar-first schema generation. If you're generating more than a few hundred records, the difference is measurable in orders of magnitude.

Quick Start

Single values

from dataforge import DataForge

forge = DataForge(locale="en_US", seed=42)

forge.person.first_name()     # "James"
forge.internet.email()        # "james.smith@gmail.com"
forge.address.city()          # "Chicago"
forge.finance.price()         # "49.99"
forge.llm.model_name()        # "gpt-4o"

Batch generation

Every method accepts count=N and returns a list:

names  = forge.person.first_name(count=1000)
emails = forge.internet.email(count=1000)
cities = forge.address.city(count=1000)

Reproducible output

forge_a = DataForge(seed=42)
forge_b = DataForge(seed=42)
assert forge_a.person.first_name() == forge_b.person.first_name()

Migrating from Faker

Change one import. Keep your existing code:

# Before
# from faker import Faker

# After — same API, ~360x faster
from dataforge.compat import Faker

fake = Faker(locale="en_US", seed=42)
fake.name()     # "James Smith"
fake.email()    # "james.smith@gmail.com"
fake.address()  # "4821 Oak Ave, Chicago, IL 60614"

57 Faker methods are mapped. Unknown methods fall back to alias lookup, then direct field matching.

Schema API

Define reusable blueprints for structured data generation:

schema = forge.schema({
    "Name": "person.full_name",
    "Email": "internet.email",
    "City": "address.city",
    "Price": "finance.price",
})

rows = schema.generate(1000)        # list[dict]
csv_str = schema.to_csv(count=5000)
sql_str = schema.to_sql(count=5000, table="users")
df = schema.to_dataframe(count=5000)  # requires pandas

Row-dependent fields for correlated data:

schema = forge.schema({
    "City": "address.city",
    "Greeting": lambda row: f"Hello from {row['City']}!",
})

Bulk Export

# List of dicts
rows = forge.to_dict(fields=["first_name", "email"], count=100)

# CSV
forge.to_csv(fields={"Name": "person.full_name"}, count=5000, path="users.csv")

# JSONL
forge.to_jsonl(fields=["first_name", "email"], count=1000, path="users.jsonl")

# SQL INSERT statements
forge.to_sql(fields=["first_name", "email"], count=500, table="users", dialect="postgresql")

# Pandas / PyArrow / Polars / Parquet (requires optional deps)
df = forge.to_dataframe(fields=["first_name"], count=10_000)
table = forge.to_arrow(fields=["first_name"], count=1_000_000)
df = forge.to_polars(fields=["first_name"], count=1_000_000)
forge.to_parquet(fields=["first_name"], path="out.parquet", count=1_000_000)

Streaming Export

For datasets that don't fit in memory:

forge.stream_to_csv(fields=["first_name", "email"], path="users.csv", count=10_000_000)
forge.stream_to_jsonl(fields=["first_name", "email"], path="users.jsonl", count=10_000_000)

CLI

Generate data from the terminal — no Python required:

dataforge --count 10 --format csv first_name last_name email
dataforge -n 100 -f jsonl -o users.jsonl first_name email city
dataforge --locale fr_FR --seed 42 -n 5 first_name city
dataforge --list-fields

Pytest Plugin

Auto-registers via pytest11 entry point. Three fixtures, zero config:

def test_name(forge):
    assert isinstance(forge.person.first_name(), str)

def test_email(fake):  # alias for forge
    assert "@" in fake.internet.email()

@pytest.mark.forge_seed(42)
def test_deterministic(forge):
    assert forge.person.first_name() == "James"

pytest --forge-seed 42  # session-wide seed

27 Providers, 198 Methods

Every method accepts count=N for batch generation.

28 Locales

forge = DataForge(locale="fr_FR")
forge.address.city()      # "Paris"
forge.person.full_name()  # "Jean Dupont"

forge = DataForge(locale="ja_JP")
forge.person.full_name()  # "田中太郎"

Multi-locale mixing:

forge = DataForge(locale=["en_US", "fr_FR", "de_DE"])
forge.person.full_name()  # randomly picks a locale per call

Unique Values

name1 = forge.unique.person.first_name()
name2 = forge.unique.person.first_name()
assert name1 != name2

names = forge.unique.person.first_name(count=100)
assert len(names) == len(set(names))

forge.unique.clear()  # reset tracking

Uses adaptive over-sampling — starts at 20% extra and scales with collision rate.

Advanced Features

from dataforge import define

schema = forge.schema({
    "Status": define(["active", "inactive", "pending"]),
    "Priority": define(["low", "medium", "high"], weights=[0.5, 0.3, 0.2]),
    "Score": define(lambda: forge.misc.boolean()),
})

from dataforge import pipe

schema = forge.schema({
    "Username": pipe("internet.username", str.upper),
    "Bio": pipe("lorem.sentence", lambda s: s[:50]),
})

from dataclasses import dataclass

@dataclass
class User:
    first_name: str
    last_name: str
    email: str

schema = forge.schema_from_dataclass(User)
rows = schema.generate(1000)

from dataforge.validation import validate

errors = validate(rows, {
    "email": {"pattern": r"^[^@]+@[^@]+\.[^@]+$"},
    "name": {"non_empty": True},
})

from dataforge.hypothesis import strategy

@given(strategy("person.full_name"))
def test_name_format(name):
    assert " " in name

dataforge --serve --port 8080
curl http://localhost:8080/api/users?count=10

schema.to_excel(count=5000, path="users.xlsx")

from dataforge.inference import fit_distribution

fit = fit_distribution(data)
# Normal, LogNormal, Exponential, Beta, Zipf

from dataforge.timeseries import TimeSeriesGenerator

ts = TimeSeriesGenerator(forge)
data = ts.generate(points=1000, trend="linear", seasonality="daily", noise=0.1)

schema = forge.infer_schema(records)
schema = forge.infer_schema_from_csv("data.csv")

from dataforge.chaos import ChaosConfig

chaos = ChaosConfig(null_rate=0.05, type_mismatch_rate=0.02)
rows = forge.with_chaos(chaos).to_dict(fields=["first_name", "email"], count=1000)

from dataforge.constraints import Constraint

schema = forge.schema({
    "City": "address.city",
    "State": Constraint("address.state", depends_on="City"),
})

from dataforge.anonymizer import Anonymizer

anon = Anonymizer(key="secret")
anon.anonymize(rows, fields=["email", "ssn"])

from dataforge.seeder import DatabaseSeeder

seeder = DatabaseSeeder(engine)
seeder.seed(User, count=10_000)

schema = forge.schema_from_openapi("openapi.json")
schema = forge.schema_from_json_schema("schema.json")

forge.stream_to_http(fields=["first_name"], url="http://api.example.com", count=1000)
forge.stream_to_kafka(fields=["first_name"], topic="users", count=1000)
forge.stream_to_rabbitmq(fields=["first_name"], queue="users", count=1000)

dataforge --tui

Integrations

PyArrow

table = forge.to_arrow(fields=["first_name", "email"], count=1_000_000)
forge.to_parquet(fields=["first_name", "email"], path="users.parquet", count=1_000_000)

Polars

df = forge.to_polars(fields=["first_name", "email"], count=1_000_000)

Pydantic

from pydantic import BaseModel

class User(BaseModel):
    first_name: str
    email: str

schema = forge.schema_from_pydantic(User)
rows = schema.generate(1000)

SQLAlchemy

schema = forge.schema_from_sqlalchemy(User)
rows = schema.generate(1000)  # primary keys auto-skipped

Benchmarks

Measured on a standard developer machine:

Run locally:

uv run python benchmark.py
uv run python benchmark.py --compare  # against saved baseline

Examples

The examples/ directory contains 21 runnable scripts covering every feature:

Installation

pip install dataforge-py        # zero dependencies
uv add dataforge-py             # with uv

Optional extras:

pip install dataforge-py[db]       # SQLAlchemy (database seeding)
pip install dataforge-py[kafka]    # confluent-kafka
pip install dataforge-py[rabbitmq] # pika
pip install dataforge-py[tui]      # textual (interactive TUI)
pip install dataforge-py[all]      # everything

Optional integrations (install separately):

pip install pyarrow polars pandas pydantic sqlalchemy openpyxl hypothesis

Requires Python >= 3.12.

Contributing

Contributions welcome. See an issue? Open one. Want to add a provider or locale? PRs appreciated.

git clone https://github.com/Sanady/dataforge-py.git
cd dataforge-py
uv sync
uv run pytest          # 1870 tests
uv run ruff check src/ tests/
uv run python benchmark.py

All commits use Conventional Commits. Performance is the primary selling point — all PRs must pass benchmark.py --compare without regressions.

If DataForge saves you time, a star helps others find it.

License

MIT