TabDat-Synth

TabDat-Synth is a Python package for spec-driven synthetic tabular data generation. It is intended for research, education, benchmarking, and data-science prototyping where real tabular data is sensitive, unavailable, or inconvenient to share.

The package learns reusable synthesis artifacts from a source table, then generates new rows from an explicit directed acyclic graph (DAG) specification. It supports empirical sampling, conditioned categorical models, numeric summaries, coefficient-based outcomes, fitted bundle reuse, and evaluation reports.

Motivation

Real tabular data often carries privacy, governance, licensing, or access constraints. These constraints can slow method development, teaching, and reproducible examples. TabDat-Synth provides a small, inspectable synthesis engine that can generate plausible tabular datasets while keeping assumptions visible in configuration.

Intended users

Features

• Spec-driven generation from YAML configuration.
• Directed synthesis workflows with stable topological execution.
• Empirical, truncated-normal, categorical-model, coefficient-sum, sigmoid-probability, and Bernoulli-response steps.
• Built-in categorical backends for LightGBM and logistic regression.
• Reusable fitted bundles for generating later without the original source file.
• Evaluation reports for marginal quality and row-level disclosure-risk heuristics.
• Small public API designed for programmatic Python workflows.

How it works

A synthesis run has five stages:

1. Load a generation specification and source table.
2. Prepare source columns according to the declared schema.
3. Fit step-level artifacts in DAG order.
4. Sample synthetic rows from the fitted artifacts.
5. Evaluate the synthetic table against source data when appropriate.

The DAG controls which generated columns are available to later steps. For categorical-model steps, declared parents define graph edges, and the fitted model receives the full incoming ancestor context in stable order.

Installation

TabDat-Synth requires Python 3.12 or newer and is distributed on PyPI.

uv add tabdat-synth

For one-off installation into an active environment:

uv pip install tabdat-synth

For workflows that load local .env files through package helpers:

uv add "tabdat-synth[env]"

Because the package is alpha software, pin downstream projects to an exact release version when reproducibility matters.

Quick start

Run the coefficient-based example from the repository root:

from tabdat_synth import generate_from_spec, load_generation_spec

spec = load_generation_spec("docs/examples/tiny_coefficient_outcomes.yaml")
df = generate_from_spec(spec)

print(df.shape)
print(df.head())

Fit once, save a reusable bundle, and generate later:

from tabdat_synth import (
  fit_synthesizer,
  generate_from_fitted,
  load_fitted_synthesizer,
  load_generation_spec,
  save_fitted_synthesizer,
)

spec = load_generation_spec("docs/examples/de_synpuf_beneficiary_phase3.yaml")
fitted = fit_synthesizer(spec)
save_fitted_synthesizer(fitted, "tmp/de_synpuf_bundle")

reloaded = load_fitted_synthesizer("tmp/de_synpuf_bundle")
synthetic = generate_from_fitted(reloaded, n_samples=20, random_seed=999)
print(synthetic.shape)

Documentation

• Getting started: installation, first run, and tests.
• Use cases: user groups, workflows, and appropriate boundaries.
• Concepts: motivation, synthesis mechanism, fitted bundles, and limitations.
• User manual: public API, step reference, backend configuration, and bundle details.
• Evaluation metrics: fidelity metrics and disclosure-risk heuristics.
• Synthpop comparison: conceptual relationship to R's synthpop package.

Acknowledgement

TabDat-Synth is conceptually inspired by R's synthpop package and the broader statistical tradition of synthetic data generation.

Nowok, B., Raab, G. M., & Dibben, C. (2016). synthpop: Bespoke creation of synthetic data in R. Journal of statistical software, 74, 1-26.

Note that this project is a cleanroom Python implementation, not built by reverse engineering synthpop, nor reuse synthpop source code. See docs/synthpop-comparison.md for a full comparison.

Public API

Common entry points:

• load_generation_spec(path)
• generate_from_spec(spec)
• fit_synthesizer(spec)
• generate_from_fitted(fitted, *, n_samples=None, random_seed=None)
• save_fitted_synthesizer(fitted, path)
• load_fitted_synthesizer(path)
• evaluate_synthetic_data(source_df, synthetic_df, schema, ...)
• evaluate_from_spec(spec, synthetic_df, ...)
• evaluation_report_to_dict(report)
• save_evaluation_report(report, path)

Testing

uv run --group dev pytest tests/unit tests/regression

License

Apache-2.0. See LICENSE.