synthext 0.1.0

LLM based synthetic dataset generator: context aware, relationship preserving and domain agnostic

synthext

LLM-powered synthetic dataset generator — LLM based synthetic dataset generator: context aware, relationship preserving and domain agnostic.

Give it a context, column names, and the required rows.
It figures out the rest — types, bounds, relationships, entities, derived columns and returns a clean pandas DataFrame in the end.

What This Library Solves

Generating synthetic data that actually makes sense is hard.

Traditional tools produce random noise - names that don't match IDs, balances that don't add up, dates are out of order, and columns with no logical relationship to each other.

synthext solves this by using LLM intelligence to:

• Understand your domain from a single context string
• Automatically classify columns as numerical or text
• Infer realistic bounds for numerical columns
• Detect entity relationships (e.g. same customer → same account number)
• Identify derived columns (e.g. running balance from transactions)
• Generate coherent, multi-batch data with chronological ordering
• Compute derived columns with exact Python arithmetic — no LLM math errors

Key Features

• Single context string — no schema definitions needed
• Automatic column classification (numerical vs text, if numerical - integer vs float)
• ER aware generation (grouped, linked, chronological)
• Derived column computation (running sums, averages, counts, min, max)
• Auto-increment columns (sequential IDs with prefix/padding)
• Batch generation with context continuity to avoid errors and hallucinations
• Domain-agnostic — works for any sort of datasets (like medial, finance, sales etc.)
• Returns pandas DataFrame
• Powered by OpenAI GPT-4o-mini

Installation

pip install synthext

Setup

Set your OpenAI API key as an environment variable:

export OPENAI_API_KEY="sk-..."

Or pass it directly:

df = Synthext(context="...", columns=[...], size=100, api_key="sk-...").generate()

Quick Start

from synthetica import Synthext

df = Synthext(
    context="Retail banking transaction records for a mid-sized private bank",
    columns=[
        "transaction_id",
        "account_number",
        "customer_name",
        "transaction_date",
        "transaction_type",
        "merchant_name",
        "transaction_amount",
        "account_balance_after_transaction"
    ],
    size=500
).generate()

print(df.head(10))
df.to_csv("banking_data.csv", index=False)

Output:

transaction_id	account_number	customer_name	transaction_date	transaction_type	merchant_name	transaction_amount	account_balance_after_transaction
TXN000001	ACCT001	Alice Johnson	2023-10-01	Deposit	Bank Transfer	5000.0	5000.0
TXN000002	ACCT001	Alice Johnson	2023-10-29	Withdrawal	ATM	1000.0	4000.0
TXN000003	ACCT001	Alice Johnson	2023-11-26	Transfer	Online Banking	150.0	3850.0
TXN000004	ACCT002	Brian Smith	2023-10-02	Deposit	Direct Deposit	2500.0	2500.0
TXN000005	ACCT002	Brian Smith	2023-10-30	Withdrawal	ATM	800.0	1700.0

Notice:

• ACCT001 always maps to Alice Johnson (entity + linked)
• transaction_id is auto-sequential (TXN000001, TXN000002, ...)
• account_balance_after_transaction is a mathematically correct running sum
• Dates are chronologically ordered per customer

Functional API

For one-liner usage:

from synthetica import generate

df = generate(
    context="E-commerce order records for an online electronics store",
    columns=["order_id", "customer_email", "customer_name", "order_date",
             "product_name", "quantity", "unit_price", "payment_method"],
    size=200
)

How It Works

The library runs a multi-stage pipeline:

Context + Columns
       │
       ▼
┌──────────────────────────┐
│ 1. Column Classification │  LLM classifies each column as numerical or text
│    (numerical vs text)   │  with subtype (integer/float)
└──────────┬───────────────┘
           ▼
┌──────────────────────────┐
│ 2. Numerical Bounds      │  LLM infers realistic min/max for each
│    Inference             │  numerical column based on domain context
└──────────┬───────────────┘
           ▼
┌──────────────────────────┐
│ 3. Relationship          │  LLM detects column roles:
│    Inference             │  entity, linked, derived, auto,
│                          │  chronological, independent
└──────────┬───────────────┘
           ▼
┌──────────────────────────┐
│ 4. Entity Generation     │  LLM creates unique entity master data
│                          │  (e.g. 30 customers with account numbers)
└──────────┬───────────────┘
           ▼
┌──────────────────────────┐
│ 5. Batch Event           │  LLM generates event rows in batches of 50
│    Generation            │  with context continuity between batches
└──────────┬───────────────┘
           ▼
┌──────────────────────────┐
│ 6. Post-Processing       │  Python sorts by entity + chronological,
│                          │  computes derived columns with exact math,
│                          │  adds auto-increment IDs
└──────────┬───────────────┘
           ▼
      pandas DataFrame

Column Roles (Auto-Detected)

The library automatically identifies these roles — no configuration needed:

Role	Description	Example
entity	Primary grouping column — the main repeating subject	customer_id, store_name, patient_id
linked	Fixed value per entity — always the same for a given entity	customer_name linked to customer_id
chronological	Time-ordered column — maintains progression per entity	transaction_date, visit_date
derived	Computed from other columns — calculated by Python, not LLM	running_balance, cumulative_spend
auto	Sequential ID — generated programmatically with prefix + padding	transaction_id → TXN000001
independent	No special relationship — generated freely by LLM	merchant_name, product_category

Derived Column Computation

Derived columns are computed by Python (not the LLM) to ensure mathematical accuracy.

Supported methods:

Method	Description	Example
running_sum	Cumulative sum per entity, with optional +/- sign logic	Account balance after each transaction
running_average	Running mean per entity	Average order value over time
running_count	Running count per entity	Total visits per patient
running_min	Running minimum per entity	Lowest score achieved
running_max	Running maximum per entity	Highest revenue recorded

Examples

Banking Transactions

from synthetica import Synthext

df = Synthext(
    context="Retail banking transaction records for a mid-sized private bank",
    columns=[
        "transaction_id",
        "account_number",
        "customer_name",
        "transaction_date",
        "transaction_type",
        "merchant_name",
        "transaction_amount",
        "account_balance_after_transaction",
        "is_fraud_suspected"
    ],
    size=100
).generate()

What the library auto-detects:

• account_number → entity
• customer_name → linked to account_number
• transaction_date → chronological
• transaction_id → auto (TXN000001, TXN000002, ...)
• account_balance_after_transaction → derived (running sum of transaction_amount, +Deposit / -Withdrawal)

E-commerce Orders

from synthetica import Synthext

df = Synthext(
    context="E-commerce order records for an online electronics and accessories store",
    columns=[
        "order_id",
        "customer_email",
        "customer_name",
        "order_date",
        "product_category",
        "product_name",
        "quantity",
        "unit_price",
        "payment_method",
        "shipping_city",
        "cumulative_customer_spend"
    ],
    size=500
).generate()

What the library auto-detects:

• customer_email → entity
• customer_name → linked to customer_email
• order_date → chronological
• order_id → auto
• cumulative_customer_spend → derived (running sum of order value per customer)

Healthcare Patient Records

from synthetica import Synthext

df = Synthext(
    context="Patient visit records for a multi-specialty hospital",
    columns=[
        "visit_id",
        "patient_id",
        "patient_name",
        "visit_date",
        "department",
        "diagnosis",
        "doctor_name",
        "bill_amount",
        "total_patient_spend"
    ],
    size=300
).generate()

What the library auto-detects:

• patient_id → entity
• patient_name → linked to patient_id
• visit_date → chronological
• visit_id → auto
• total_patient_spend → derived (running sum of bill_amount per patient)

Sales Pipeline (CRM)

from synthetica import Synthext

df = Synthext(
    context="B2B sales pipeline data for a SaaS company",
    columns=[
        "deal_id",
        "company_name",
        "contact_person",
        "stage",
        "deal_value",
        "close_date",
        "sales_rep"
    ],
    size=200
).generate()

Simple Flat Dataset (No Entities)

Not every dataset has entity relationships. The library handles flat datasets too:

from synthetica import Synthext

df = Synthext(
    context="Used car listings from an online marketplace",
    columns=[
        "listing_id",
        "car_brand",
        "model",
        "year",
        "mileage_km",
        "fuel_type",
        "transmission",
        "price_usd",
        "seller_city"
    ],
    size=500
).generate()

When no entity relationships are detected, the library falls back to a simpler generation mode — still batched and context-aware, but without entity grouping or derived columns.

Parameters

Parameter	Type	Required	Default	Description
context	str	Yes	—	Domain description for the dataset
columns	list[str]	Yes	—	Column names
size	int	Yes	—	Number of rows to generate
api_key	str	No	OPENAI_API_KEY env var	OpenAI API key

Output

Returns a standard pandas.DataFrame with:

• All requested columns in the original order
• Correct data types (numerical columns as float/int, text as string)
• Entity consistency across all rows
• Mathematically correct derived columns
• Sequential auto-increment IDs

Requirements

• Python >= 3.9
• OpenAI API key (GPT-4o-mini)

Contributing

Pull requests are welcome.

For major changes, please open an issue first to discuss what you would like to change.

Please ensure tests and documentation are updated where applicable.

License

MIT License