Skip to main content

Tools for evaluating fidelity and privacy of synthetic data

Project description


This library contains tools for evaluating fidelity and privacy of synthetic data.


Import the desired modules from the library:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from tonic_reporting import univariate, multivariate, privacy


Numeric columns refer to columns encoded as numeric. Numerical data types in the schema underlying a model may be encoded as other types.

Categorical columns refer to columns encoded as categorical.

source_df is a Pandas DataFrame of original data from the source database

synth_df is a Pandas DataFrame of sampled data from trained models

The source and synthetic DataFrames should be equal in row count and schema.

Numeric Column Statistics

univariate.summarize_numeric(source_df, synth_df, numeric_cols)

Categorical Column Statistics

univariate.summarize_categorical(source_df, synth_df, categorical_cols)

Numeric Column Comparative Histograms

fig, axarr = plt.subplots(1, len(numeric_cols), figsize = (9,12))
axarr = axarr.ravel()

for col, ax in zip(numeric_cols, axarr):
    univariate.plot_histogram(source_df, synth_df, col,ax)

Categorical Column Comparative Frequency Tables

for col in categorical_cols:
    univariate.plot_frequency_table(source_df, synth_df, col, ax)

Numeric Column Aggregates Over Time

If the data represents time series, we can visualize means and confidence intervals of numeric features over time:

for col in numeric_cols:
    fig, ax = plt.subplots(figsize=(10, 8))
    univariate.plot_events_means(source_df, synth_df, col, order_col, ax=ax)


for col in numeric_cols:
    fig, ax = plt.subplots(figsize=(12, 10))
    univariate.plot_events_confidence_intervals(source_df, synth_df, col, order_col, ax=ax)

where order_col denotes the time/order column.

Numeric Column Multivariate Correlations Table

multivariate.summarize_correlations(source_df, synth_df, numeric_cols)

Numeric Column Multivariate Correlations Heat Map

fig, axarr = plt.subplots(1, 2, figsize=(13, 8))
multivariate.plot_correlations(source_df, synth_df, numeric_cols, axarr=axarr, )

Distance to Closest Record Comparison

syn_dcr, real_dcr = privacy.compute_dcr(source_df, synth_df, numeric_cols, categorical_cols)

fig, ax = plt.subplots(1,1,figsize=(8,6))
ax.hist(real_dcr,bins=300,label = 'Real vs. real', color='mediumpurple');
ax.hist(syn_dcr,bins=300,label='Synthetic vs. real', color='mediumturquoise');
ax.tick_params(axis='both', which='major', labelsize=14)
ax.set_title('Distances to closest record',fontsize=22)

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tonic_reporting-1.5.0.tar.gz (10.2 kB view hashes)

Uploaded source

Built Distribution

tonic_reporting-1.5.0-py3-none-any.whl (11.6 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page