Synthetic Data Quality Assurance 🔎
Documentation | Sample Reports | Technical White Paper
Assess the fidelity and novelty of synthetic samples with respect to original samples:
- calculate a rich set of accuracy, similarity and distance metrics
- visualize statistics for easy comparison to training and holdout samples
- generate a standalone, easy-to-share, easy-to-read HTML summary report
...all with a few lines of Python code 💥.
https://github.com/user-attachments/assets/b27e270a-f19c-4059-b4f2-ed209c9a26b9
Installation
The latest release of mostlyai-qa can be installed via pip:
pip install -U mostlyai-qa
On Linux, one can explicitly install the CPU-only variant of torch together with mostlyai-qa:
pip install -U torch==2.13.0+cpu torchvision==0.28.0+cpu mostlyai-qa --extra-index-url https://download.pytorch.org/whl/cpu
Quick Start
import pandas as pd
import webbrowser
from mostlyai import qa
# initialize logging to stdout
qa.init_logging()
# fetch original + synthetic data
base_url = "https://github.com/mostly-ai/mostlyai-qa/raw/refs/heads/main/examples/quick-start"
syn = pd.read_csv(f"{base_url}/census2k-syn_mostly.csv.gz")
# syn = pd.read_csv(f'{base_url}/census2k-syn_flip30.csv.gz') # a 30% perturbation of trn
trn = pd.read_csv(f"{base_url}/census2k-trn.csv.gz")
hol = pd.read_csv(f"{base_url}/census2k-hol.csv.gz")
# calculate metrics
report_path, metrics = qa.report(
syn_tgt_data=syn,
trn_tgt_data=trn,
hol_tgt_data=hol,
)
# pretty print metrics
print(metrics.model_dump_json(indent=4))
# open up HTML report in new browser window
webbrowser.open(f"file://{report_path.absolute()}")
Basic Usage
from mostlyai import qa
# initialize logging to stdout
qa.init_logging()
# analyze single-table data
report_path, metrics = qa.report(
syn_tgt_data = synthetic_df,
trn_tgt_data = training_df,
hol_tgt_data = holdout_df, # optional
)
# analyze sequential data
report_path, metrics = qa.report(
syn_tgt_data = synthetic_df,
trn_tgt_data = training_df,
hol_tgt_data = holdout_df, # optional
tgt_context_key = "user_id",
)
# analyze sequential data with context
report_path, metrics = qa.report(
syn_tgt_data = synthetic_df,
trn_tgt_data = training_df,
hol_tgt_data = holdout_df, # optional
syn_ctx_data = synthetic_context_df,
trn_ctx_data = training_context_df,
hol_ctx_data = holdout_context_df, # optional
ctx_primary_key = "id",
tgt_context_key = "user_id",
)
Sample Reports
- Baseball Players (Flat Data)
- Baseball Seasons (Sequential Data)
Citation
Please consider citing our project if you find it useful:
@misc{mostlyai-qa,
title={Benchmarking Synthetic Tabular Data: A Multi-Dimensional Evaluation Framework},
author={Andrey Sidorenko and Michael Platzer and Mario Scriminaci and Paul Tiwald},
year={2025},
eprint={2504.01908},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2504.01908},
}
2 maintainers
Metadata
2 maintainers
Release files for mostlyai-qa 1.10.8
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| mostlyai_qa-1.10.8.tar.gz | 30.6 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| mostlyai_qa-1.10.8-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 61.3 MB
Release files / mostlyai_qa-1.10.8.tar.gz
| Download URL | mostlyai_qa-1.10.8.tar.gz |
|---|---|
| Size | 30.6 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ddf3924d121dc6a81a449f73247fba1b9517a52acd674aa22aac164ec8891947
|
|
BLAKE2b-256 checksum How to use checksums |
6e1b578ca0f9d9af4efe05de5530904d39a9340a636f38221bd939913a1d0492
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 6, 2026.
Transparency logRelease files / mostlyai_qa-1.10.8-py3-none-any.whl
| Download URL | mostlyai_qa-1.10.8-py3-none-any.whl |
|---|---|
| Size | 30.7 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
de0a80b9fe1949cd5b535394b1ab694415d2afb1ab9327bcb65a783801949419
|
|
BLAKE2b-256 checksum How to use checksums |
c016a4453ffc4486767b929025e55c9596b912582c9fa1dbcd9514e91df9d696
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 6, 2026.
Transparency log
