👯 DP-CGANS (Differentially Private - Conditional Generative Adversarial NetworkS)

Abstract: This repository presents a Conditional Generative Adversary Networks (GANs) on tabular data (and RDF data) combining with Differential Privacy techniques. Our pre-print publication: Generating synthetic personal health data using conditional generative adversarial networks combining with differential privacy.

Author: Chang Sun, Institute of Data Science, Maastricht University Start date: Nov-2021 Status: Under development

Note: "Standing on the shoulders of giants". This repository is inspired by the excellent work of CTGAN from Synthetic Data Vault (SDV), Tensorflow Privacy, and RdfPdans. Highly appreciate they shared the ideas and implementations, made code publicly available, well-written documentation. More related work can be found in the References below.

This package is extended from SDV (https://github.com/sdv-dev/SDV), CTGAN (https://github.com/sdv-dev/CTGAN), and Differential Privacy in GANs (https://github.com/civisanalytics/dpwgan). The author modified the conditional matrix and cost functions to emphasize on the relations between variables. The main changes are in ctgan/synthesizers/ctgan.py ../data_sampler.py ../data_transformer.py

🎯 What to use DP-CGANS for

DP-CGANS learns the joint distribution of a tabular dataset and generates new rows that follow it, optionally under a differential-privacy guarantee. The table lists the purposes the model is built for and how the package supports each one.

Purpose	What it means	How DP-CGANS supports it
Open release	The synthetic output is shared or published in place of the real data.	Train with private=True: the discriminator is trained with DP-SGD and the spent privacy budget (ε, δ) is reported with Rényi DP accounting, so the generated data can be released as a privacy-preserving substitute.
Pipeline testing	Exercising software, formats or pipelines; no realism claim.	Any CSV goes in, a CSV with the same columns and types comes out. dp-cgans gen produces development data in one command.
ML augmentation	Adding synthetic rows to train a predictive model.	Sample as many rows as you need with model.sample(n). The paper evaluates train-on-synthetic, test-on-real performance.
Statistical replication	The synthetic data reproduces the real distributions and relationships closely enough to rerun an analysis on it.	The conditional vector and loss functions were modified to emphasise the relations between variables. The paper evaluates marginal distributions, pairwise correlations and statistical similarity against the real data.
Causal inference	The synthetic data preserves a known or specified causal structure.	DP-CGANS fits a joint distribution; it does not encode a causal graph. Use it to exercise a causal-analysis pipeline, not to validate causal claims.
Education	Synthetic data used as teaching material, for demonstrations or in training courses.	Generate a realistic but non-identifiable copy of a sensitive dataset so students can practise on it, for example medical students learning statistical analysis on synthetic patient records. Trained with private=True, the teaching data carries a formal privacy guarantee and can be handed out freely.
Benchmarking	Used as a baseline or benchmark dataset for comparing generators or metrics.	A published method with a pip-installable implementation, a fixed example dataset and a CLI, so it can serve as a reproducible baseline in generator comparisons.
Imbalance correction	Oversampling a rare class.	model.sample(n, condition_column=..., condition_value=...) draws rows for a chosen category, so a minority class can be oversampled directly.

📥️ Installation

You will need Python >=3.8+

pip3 install -U dp-cgans

🪄 Usage

🐍 Use with Python

Use the library directly in Python scripts when you want full control over the model hyper-parameters, the differential-privacy setting (private=True), and conditional sampling.

New to DP-CGANS? Try it first in the tutorial notebook on Colab, which runs the example below step by step in your browser.

If your input is tabular data (e.g., csv):

from dp_cgans import DP_CGAN
import pandas as pd
import time

input_file = "../resources/example_tabular_data_UCIAdult.csv"
tabular_data = pd.read_csv(input_file)

### Add your pre-processing if needed
for col in tabular_data.columns:
  tabular_data[col] = pd.to_numeric(tabular_data[col], errors='ignore', downcast='integer')
for col in tabular_data.columns:
  if tabular_data[col].nunique() < 10: 
      tabular_data[col] = tabular_data[col].astype('object')

# Configure model hyper-parameters
model = DP_CGAN(
  epochs=100, # number of training epochs
  batch_size=100, # the size of each batch
  log_frequency=True,
  verbose=True,
  generator_dim=(128, 128, 128),
  discriminator_dim=(128, 128, 128),
  generator_lr=2e-4, 
  discriminator_lr=2e-4,
  discriminator_steps=10, 
  private=False,
  saved_transformer=None #"fitted_transformer.pkl"
)

start_time = time.time()
print("Start training the model: ")
model.fit(tabular_data)
end_time = time.time()

elapsed_time = end_time - start_time
print("Training model time ", elapsed_time)

print("Saving the trained generator...")
model.save("generator.pkl")

# print("load the trained file.")
# loaded_model=DP_CGAN.load("PATH_TO_MODEL")

# Generate 100 synthetic rows.
# Numeric columns come back with the same dtype and the same number of
# decimal places as the real data (e.g. 2 decimals in, 2 decimals out).
syn_data = model.sample(100)

syn_data.to_csv("syn_data_file.csv",index=None)

⌨️ Use as a command-line interface

If you only need a quick synthetic copy of a CSV file, you can generate it from your terminal after installing dp-cgans with pip (default hyper-parameters, no differential privacy).

To quickly run our example, you can download the example data:

wget https://raw.githubusercontent.com/sunchang0124/dp_cgans/main/resources/example_tabular_data_UCIAdult.csv

Then run dp-cgans:

dp-cgans gen example_tabular_data_UCIAdult.csv --epochs 100 --output out.csv --gen-size 100

Get a full rundown of the available options for generating synthetic data with:

dp-cgans gen --help

📓 Tutorial notebook

A hands-on tutorial that walks through loading a dataset, training DP-CGANS, sampling synthetic rows, and inspecting the result runs in the browser, no installation needed:

DP-CGANS hands-on notebook (source: sunchang0124/syndata-workshop)

🧑‍💻 Development setup

For development, we recommend to install and use Hatch, as it will automatically install and sync the dependencies when running development scripts. But you can also directly create a virtual environment and install the library with pip install -e .

Install

Clone the repository:

git clone https://github.com/sunchang0124/dp_cgans
cd dp_cgans

When working in development the hatch tool will automatically install and sync the dependencies when running a script. But you can also directly

Run

Run the library with the CLI:

hatch -v run dp-cgans gen --help

You can also enter a new shell with the virtual environments automatically activated:

hatch shell
dp-cgans gen --help

Tests

Run the tests locally:

hatch run pytest -s

Format

Run formatting and linting (black and ruff):

hatch run fmt

Reset the virtual environments

In case the virtual environments is not updating as expected you can easily reset it with:

hatch env prune

📦️ New release process

The deployment of new releases is done automatically by a GitHub Action workflow when a new release is created on GitHub. To release a new version:

1. Make sure the PYPI_API_TOKEN secret has been defined in the GitHub repository (in Settings > Secrets > Actions). You can get an API token from PyPI here.

2. Increment the version number in src/dp_cgans/__init__.py file:

   hatch version fix    # Bump from 0.0.1 to 0.0.2
   hatch version minor  # Bump from 0.0.1 to 0.1.0
   hatch version 0.1.1  # Bump to the specified version
   

3. Create a new release on GitHub, which will automatically trigger the publish workflow, and publish the new release to PyPI.

You can also manually build and publish from you laptop:

hatch build
hatch publish

📚️ References / Further reading

There are many excellent work on generating synthetic data using GANS and other methods. We list the studies that made great conbributions for the field and inspiring for our work.

GANS

1. Synthetic Data Vault (SDV) [Paper] [Github]
2. Modeling Tabular Data using Conditional GAN (a part of SDV) [Paper] [Github]
3. Wasserstein GAN [Paper]
4. Improved Training of Wasserstein GANs [Paper]
5. Synthesising Tabular Data using Wasserstein Conditional GANs with Gradient Penalty (WCGAN-GP) [Paper]
6. PacGAN: The power of two samples in generative adversarial networks [Paper]
7. CTAB-GAN: Effective Table Data Synthesizing [Paper]
8. Conditional Tabular GAN-Based Two-Stage Data Generation Scheme for Short-Term Load Forecasting [Paper]
9. TabFairGAN: Fair Tabular Data Generation with Generative Adversarial Networks [Paper]
10. Conditional Wasserstein GAN-based Oversampling of Tabular Data for Imbalanced Learning [Paper]

Differential Privacy

1. Tensorflow Privacy [Github]
2. Renyi Differential Privacy [Paper]
3. DP-CGAN : Differentially Private Synthetic Data and Label Generation [Paper]
4. Differentially Private Generative Adversarial Network [Paper] [Github] Another implementation [Github]
5. Private Data Generation Toolbox [Github]
6. autodp: Automating differential privacy computation [Github]
7. Differentially Private Synthetic Medical Data Generation using Convolutional GANs [Paper]
8. DTGAN: Differential Private Training for Tabular GANs [Paper]
9. DIFFERENTIALLY PRIVATE SYNTHETIC DATA: APPLIED EVALUATIONS AND ENHANCEMENTS [Paper]
10. FFPDG: FAST, FAIR AND PRIVATE DATA GENERATION [Paper]

Others

1. EvoGen: a Generator for Synthetic Versioned RDF [Paper]
2. Generation and evaluation of synthetic patient data [Paper]
3. Fake It Till You Make It: Guidelines for Effective Synthetic Data Generation [Paper]
4. Generating and evaluating cross-sectional synthetic electronic healthcare data: Preserving data utility and patient privacy [Paper]
5. Synthetic data for open and reproducible methodological research in social sciences and official statistics [Paper]
6. A Study of the Impact of Synthetic Data Generation Techniques on Data Utility using the 1991 UK Samples of Anonymised Records [Paper]