Skip to main content

The Synthetic Data API — privacy-preserving synthetic data generation

Project description

DataXID Python SDK

PyPI version Python versions License

High-fidelity synthetic data generation for single-table, multi-table, and time series data.

Installation

pip install dataxid

Quick Start

import dataxid
import pandas as pd

dataxid.api_key = "dx_..."
dataxid.enable_logging("info")  # optional: see training progress

df = pd.read_csv("data.csv")
synthetic = dataxid.synthesize(data=df, n_samples=1000)

Multi-Table & Time Series

Synthesize related tables with referential integrity. Child tables are generated sequentially by default — preserving realistic per-entity patterns like transaction counts, temporal ordering, and sequence lengths.

from dataxid import Table

accounts = Table(pd.read_csv("accounts.csv"), primary_key="account_id")
transactions = Table(pd.read_csv("transactions.csv"),
                     foreign_keys={"account_id": accounts})

synthetic = dataxid.synthesize_tables({
    "accounts": accounts,
    "transactions": transactions,
})

synthetic["accounts"]       # synthetic accounts with auto-assigned PKs
synthetic["transactions"]   # sequential transactions per account, valid FKs

Per-table generation controls are passed as dicts keyed by table name:

from dataxid import Synthetic, Distribution

accounts_preset = Synthetic(n=1000)
transactions_preset = Synthetic(n=5000, seed=42)

country_distribution = Distribution(
    column="country",
    probabilities={"US": 0.6, "UK": 0.4},
)

synthetic = dataxid.synthesize_tables(
    tables={"accounts": accounts, "transactions": transactions},
    synthetic={
        "accounts": accounts_preset,
        "transactions": transactions_preset,
    },
    distribution={"accounts": country_distribution},
)

Iterative workflow

When you want to train once and generate many times — for example, running several sampling strategies against the same model — split the call into Model.create and model.generate:

model = dataxid.Model.create(data=df)
synthetic_a = model.generate(n_samples=1000, diversity=0.8)
synthetic_b = model.generate(n_samples=1000, diversity=1.2)
model.delete()

How It Works

DataXID is built on a privacy-by-architecture principle. Data encoding and decoding happen entirely on your machine; only abstract embeddings are shared with the API for model training. Raw data never leaves your environment.

Configuration

Parameter Default Description
embedding_dim 64 Embedding size (larger = more expressive)
model_size "medium" Model capacity: "small", "medium", "large"
max_epochs 100 Maximum training epochs
batch_size 256 Training batch size
privacy Privacy() Privacy config (see below)
config = dataxid.ModelConfig(
    embedding_dim=128,
    model_size="large",
    max_epochs=50,
    privacy=dataxid.Privacy(enabled=True, noise=0.2),
)
model = dataxid.Model.create(data=df, config=config)

A plain dict is also accepted for quick experiments: config={"embedding_dim": 128}.

Privacy

Privacy settings are grouped under a dedicated Privacy config:

Parameter Default Description
enabled False Add Gaussian noise to embeddings before they leave the machine
noise 0.1 Noise scale (Gaussian std) when enabled=True
protect_rare True Hide rare categorical values behind a <protected> token
rare_strategy "mask" How protected values appear: "mask" or "sample"

Advanced Generation

Rebalance category distributions

Override the natural distribution of a categorical column:

model = dataxid.Model.create(data=df)

distribution = dataxid.Distribution(
    column="gender",
    probabilities={"M": 0.5, "F": 0.5},
)
synthetic = model.generate(n_samples=1000, distribution=distribution)

Bias correction

Reduce statistical parity gaps across sensitive attributes:

bias = dataxid.Bias(
    target="income",
    sensitive=["gender", "race"],
)
synthetic = model.generate(n_samples=1000, bias=bias)

Conditional generation

Fix known values and let the model complete the rest:

conditions = pd.DataFrame({"income": [">50K"] * 1000})

synthetic = model.generate(conditions=conditions)

Impute missing values

Fill NaN cells with model predictions; non-NULL cells are preserved:

model = dataxid.Model.create(data=df)
filled = model.impute(df, trials=3, pick="mode")

Tuning presets

Bundle generation-time knobs into a reusable preset:

preset = dataxid.Synthetic(
    n=1000,
    seed=42,
    diversity=0.8,
    rare_cutoff=0.95,
)
synthetic = model.generate(synthetic=preset)

Direct keyword arguments override preset values:

synthetic = model.generate(synthetic=preset, diversity=1.2)  # diversity=1.2 wins

Logging

dataxid.enable_logging("info")   # see training progress, epoch stats
dataxid.enable_logging("debug")  # verbose: includes HTTP requests
dataxid.disable_logging()        # turn off (default state)

Or via environment variable (no code change needed):

DATAXID_LOG=info python my_script.py

Error Handling

import dataxid

try:
    synthetic = dataxid.synthesize(data=df)
except dataxid.AuthenticationError:
    print("Invalid API key")
except dataxid.QuotaExceededError as e:
    print(f"Quota exceeded. Upgrade: {e.upgrade_url}")
except dataxid.RateLimitError as e:
    print(f"Rate limited. Retry after: {e.retry_after}s")
except dataxid.DataxidError as e:
    print(f"Error: {e}")

Links

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dataxid-0.3.1.tar.gz (176.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dataxid-0.3.1-py3-none-any.whl (76.6 kB view details)

Uploaded Python 3

File details

Details for the file dataxid-0.3.1.tar.gz.

File metadata

  • Download URL: dataxid-0.3.1.tar.gz
  • Upload date:
  • Size: 176.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.9.22 {"installer":{"name":"uv","version":"0.9.22","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for dataxid-0.3.1.tar.gz
Algorithm Hash digest
SHA256 5a552088b8cdb0b88120a212ff74002d42d78dbf6348ce862aea08f4eda8ba4b
MD5 84f93ed5898a99ee6bd22cd81434cf58
BLAKE2b-256 a4c8868f9eedc9ff6a398a511a753c1310f5bf62a191ff65d89fd00e55700d3b

See more details on using hashes here.

File details

Details for the file dataxid-0.3.1-py3-none-any.whl.

File metadata

  • Download URL: dataxid-0.3.1-py3-none-any.whl
  • Upload date:
  • Size: 76.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.9.22 {"installer":{"name":"uv","version":"0.9.22","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for dataxid-0.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 53282458269f2557d4bacdb47920e3a59dedc11f2dd6d2ccecf88213dde24d7b
MD5 be33e5e0c4b0b5fc597cba637af2eee0
BLAKE2b-256 eb8fe63c45bef8105e731611241ec56d9c3f38f6e4cb50828c8a1492f6d4f793

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page