Skip to main content

This repository is part of The Synthetic Data Vault Project, a project from DataCebo.

Dev Status PyPi Shield Unit Tests Integration Tests Coverage Status Downloads Colab Forum

Overview

The Synthetic Data Vault (SDV) is a Python library designed to be your one-stop shop for creating tabular synthetic data. The SDV uses a variety of machine learning algorithms to learn patterns from your real data and emulate them in synthetic data.

Features

:brain: Create synthetic data using machine learning. The SDV offers multiple models, ranging from classical statistical methods (GaussianCopula) to deep learning methods (CTGAN). Generate data for single tables, multiple connected tables or sequential tables.

:bar_chart: Evaluate and visualize data. Compare the synthetic data to the real data against a variety of measures. Diagnose problems and generate a quality report to get more insights.

:arrows_counterclockwise: Preprocess, anonymize and define constraints. Control data processing to improve the quality of synthetic data, choose from different types of anonymization and define business rules in the form of logical constraints.

Important Links
Tutorials Get some hands-on experience with the SDV. Launch the tutorial notebooks and run the code yourself.
:book: Docs Learn how to use the SDV library with user guides and API references.
:orange_book: Blog Get more insights about using the SDV, deploying models and our synthetic data community.
:busts_in_silhouette: DataCebo Forum Discuss SDV features, ask questions, and receive help .
:computer: Website Check out the SDV website for more information about the project.

Install

The SDV is publicly available under the Business Source License. Install SDV using pip or conda. We recommend using a virtual environment to avoid conflicts with other software on your device.

pip install sdv
conda install -c pytorch -c conda-forge sdv

Getting Started

Load a demo dataset to get started. This dataset is a single table describing guests staying at a fictional hotel.

from sdv.datasets.demo import download_demo

real_data, metadata = download_demo(modality='single_table', dataset_name='fake_hotel_guests')

Single Table Metadata Example

The demo also includes metadata, a description of the dataset, including the data types in each column and the primary key (guest_email).

Synthesizing Data

Next, we can create an SDV synthesizer, an object that you can use to create synthetic data. It learns patterns from the real data and replicates them to generate synthetic data. Let's use the GaussianCopulaSynthesizer.

from sdv.single_table import GaussianCopulaSynthesizer

synthesizer = GaussianCopulaSynthesizer(metadata)
synthesizer.fit(data=real_data)

And now the synthesizer is ready to create synthetic data!

synthetic_data = synthesizer.sample(num_rows=500)

The synthetic data will have the following properties:

  • Sensitive columns are fully anonymized. The email, billing address and credit card number columns contain new data so you don't expose the real values.
  • Other columns follow statistical patterns. For example, the proportion of room types, the distribution of check in dates and the correlations between room rate and room type are preserved.
  • Keys and other relationships are intact. The primary key (guest email) is unique for each row. If you have multiple tables, the connection between a primary and foreign keys makes sense.

Evaluating Synthetic Data

The SDV library allows you to evaluate the synthetic data by comparing it to the real data. Get started by generating a quality report.

from sdv.evaluation.single_table import evaluate_quality

quality_report = evaluate_quality(real_data, synthetic_data, metadata)
Generating report ...

(1/2) Evaluating Column Shapes: |████████████████| 9/9 [00:00<00:00, 1133.09it/s]|
Column Shapes Score: 89.11%

(2/2) Evaluating Column Pair Trends: |██████████████████████████████████████████| 36/36 [00:00<00:00, 502.88it/s]|
Column Pair Trends Score: 88.3%

Overall Score (Average): 88.7%

This object computes an overall quality score on a scale of 0 to 100% (100 being the best) as well as detailed breakdowns. For more insights, you can also visualize the synthetic vs. real data.

from sdv.evaluation.single_table import get_column_plot

fig = get_column_plot(
    real_data=real_data,
    synthetic_data=synthetic_data,
    column_name='amenities_fee',
    metadata=metadata,
)

fig.show()

Real vs. Synthetic Data

What's Next?

Using the SDV library, you can synthesize single table, multi table and sequential data. You can also customize the full synthetic data workflow, including preprocessing, anonymization and adding constraints.

To learn more, visit the SDV Demo page.

Credits

Thank you to our team of contributors who have built and maintained the SDV ecosystem over the years!

View Contributors

Citation

If you use SDV for your research, please cite the following paper:

Neha Patki, Roy Wedge, Kalyan Veeramachaneni. The Synthetic Data Vault. IEEE DSAA 2016.

@inproceedings{
    SDV,
    title={The Synthetic data vault},
    author={Patki, Neha and Wedge, Roy and Veeramachaneni, Kalyan},
    booktitle={IEEE International Conference on Data Science and Advanced Analytics (DSAA)},
    year={2016},
    pages={399-410},
    doi={10.1109/DSAA.2016.49},
    month={Oct}
}



The Synthetic Data Vault Project was first created at MIT's Data to AI Lab in 2016. After 4 years of research and traction with enterprise, we created DataCebo in 2020 with the goal of growing the project. Today, DataCebo is the proud developer of SDV, the largest ecosystem for synthetic data generation & evaluation. It is home to multiple libraries that support synthetic data, including:

  • 🔄 Data discovery & transformation. Reverse the transforms to reproduce realistic data.
  • 🧠 Multiple machine learning models -- ranging from Copulas to Deep Learning -- to create tabular, multi table and time series data.
  • 📊 Measuring quality and privacy of synthetic data, and comparing different synthetic data generation models.

Get started using the SDV package -- a fully integrated solution and your one-stop shop for synthetic data. Or, use the standalone libraries for specific needs.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sdv-1.38.2.tar.gz (185.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

sdv-1.38.2-py3-none-any.whl (213.9 kB view details)

Uploaded Python 3

File details

Details for the file sdv-1.38.2.tar.gz.

File metadata

  • Download URL: sdv-1.38.2.tar.gz
  • Upload date:
  • Size: 185.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for sdv-1.38.2.tar.gz
Algorithm Hash digest
SHA256 5881627b7e40ce6385a14c6b6d0729701f872117c38f7cb1c82933e00fb76062
MD5 dbb8f9043a517f21f81e7777f4278eb4
BLAKE2b-256 f613038d9944eff022fe72020f2527117732632f04e39ae0e8547218faf8e85b

See more details on using hashes here.

Provenance

The following attestation bundles were made for sdv-1.38.2.tar.gz:

Publisher: release.yml on sdv-dev/SDV

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file sdv-1.38.2-py3-none-any.whl.

File metadata

  • Download URL: sdv-1.38.2-py3-none-any.whl
  • Upload date:
  • Size: 213.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for sdv-1.38.2-py3-none-any.whl
Algorithm Hash digest
SHA256 b094084aa901bf986f38c3e0b45bd0d66b7299d9945fee2b370466d1fe007ad8
MD5 0fecd6ac61576adfcc2c2af35023a0a9
BLAKE2b-256 31b8c412b07be2834f408c6acdbc6ae85f93707bbfc5fa84d27b7adb6ae20d74

See more details on using hashes here.

Provenance

The following attestation bundles were made for sdv-1.38.2-py3-none-any.whl:

Publisher: release.yml on sdv-dev/SDV

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

1.38.3

2 files

This release

1.38.2 This release

2 files

1.38.1

2 files

1.38.0

2 files

1.37.4

2 files

1.37.3

2 files

1.37.2

2 files

1.37.1

2 files

1.37.0

2 files

1.36.3

2 files

1.36.2

2 files

1.36.1

2 files

1.36.0

2 files

1.35.1

2 files

1.35.0

2 files

1.34.3

2 files

1.34.2

2 files

1.34.1

2 files

1.34.0

2 files

1.33.1

2 files

1.33.0

2 files

1.32.1

2 files

1.32.0

2 files

1.31.0

2 files

1.30.0

2 files

1.29.1

2 files

1.29.0

2 files

1.28.0

2 files

1.27.0

2 files

1.26.0

2 files

1.25.0

2 files

1.24.1

2 files

1.24.0

2 files

1.23.0

2 files

1.22.1

2 files

1.22.0

2 files

1.21.0

2 files

1.20.1

2 files

1.20.0

2 files

1.19.0

2 files

1.18.0

2 files

1.17.4

2 files

1.17.3

2 files

1.17.2

2 files

1.17.1

2 files

1.17.0

2 files

1.16.2

2 files

1.16.1

2 files

1.16.0

2 files

1.15.0

2 files

1.14.0

2 files

1.13.1

2 files

1.13.0

2 files

1.12.1

2 files

1.12.0

2 files

1.11.0

2 files

1.10.0

2 files

1.9.0

2 files

1.8.0

2 files

1.7.0

2 files

1.6.0

2 files

1.5.0

2 files

1.4.0

2 files

1.3.0

2 files

1.2.1

2 files

1.2.0

2 files

1.1.0

2 files

1.0.1

2 files

1.0.0

2 files

0.18.0

2 files

0.17.2

2 files

0.17.1

2 files

0.17.0

2 files

0.16.0

2 files

0.15.0

2 files

0.14.1

2 files

0.14.0

2 files

0.13.1

2 files

0.13.0

2 files

0.12.1

2 files

0.12.0

2 files

0.11.0

2 files

0.10.1

2 files

0.10.0

2 files

0.9.1

2 files

0.9.0

2 files

0.8.0

2 files

0.7.0

2 files

0.6.1

2 files

0.6.0

2 files

0.5.0

2 files

0.4.5

2 files

0.4.4

2 files

0.4.3

2 files

0.4.2

2 files

0.4.1

2 files

0.4.0

2 files

0.3.6

2 files

0.3.5

2 files

0.3.4

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.2

2 files

0.1.1

2 files

0.1.0

2 files

0.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page