synthia 1.0.0

Multi-dimensional synthetic data generation in Python

Overview | Documentation | How to cite | Contributing | Development notes | Copyright and license | Acknowledgements

Overview

Synthetic data need to preserve the statistical properties of real data in terms of their individual behavior and (inter-)dependences (Meyer et al. 2021). Copula and functional Principle Component Analysis (fPCA) are statistical models that allow these properties to be simulated (Joe 2014). As such, copula generated data have shown potential to improve the generalization of machine learning (ML) emulators (Meyer et al. 2021) or anonymize real-data datasets (Patki et al. 2016).

Synthia is an open source Python package to model univariate and multivariate data, parameterize data using empirical and parametric methods, and manipulate marginal distributions. It is designed to enable scientists and practitioners to handle labelled multivariate data typical of computational sciences. For example, given some vertical profiles of atmospheric temperature, we can use Synthia to generate new but statistically similar profiles in just three lines of code (Table 1).

Synthia supports three methods of multivariate data generation through fPCA, parametric (Gaussian) copula, and vine copula models for continuous (all), discrete (vine), and categorical (vine) variables. It has a simple and succinct API to natively handle xarray's labelled arrays and datasets. It uses a pure Python implementation for fPCA and Gaussian copula, and relies on the fast and well tested C++ library vinecopulib through pyvinecopulib's bindings for fast and efficient computation of vines. For more information about the latest version of Synthia, please see the website.

Table 1. Example application of Gaussian and fPCA classes in Synthia. These are used to generate random profiles of atmospheric temperature similar to those included in the source data. The xarray dataset structure is maintained and returned by Synthia.

Source	Synthetic with Gaussian Copula	Synthetic with fPCA
ds = syn.util.load_dataset()	g = syn.CopulaDataGenerator()	g = syn.fPCADataGenerator()
	g.fit(ds, syn.GaussianCopula())	g.fit(ds)
	g.generate(n_samples=500)	g.generate(n_samples=500)

Documentation

For installation instructions, getting started guides and tutorials, background information, and API reference summaries, please see the website.

How to cite

When using Synthia, please cite the following two papers using the following Digital Object Identifiers (DOIs). You can generate citations in your preferred style with DOI Citation Formatter or use the BibTeX entries below.

Application	Software
10.5194/gmd-2020-427	10.21105/joss.02863

@article{Meyer2021,
  doi       = {10.5194/gmd-2020-427},
  year      = {2021},
  publisher = {Copernicus {GmbH}},
  author    = {David Meyer and Thomas Nagler and Robin J. Hogan},
  title     = {Copula-based synthetic data generation for machine learning emulators
		in weather and climate: application to a simple radiation model},
  note = {Under review}
}


@article{MeyerAndNagler2021,
  title = {Synthia: multidimensional synthetic data generation in Python},
  author = {David Meyer and Thomas Nagler},
  year = {2021},
  doi = {10.21105/joss.02863},
  journal = {Journal of Open Source Software},
  note = {Under review}
}

Note: For reproducibility we recommend to cite the specific version of the software used.

Contributing

If you are looking to contribute, please read our Contributors' guide for details.

Development notes

If you would like to know more about specific development guidelines, testing and deployment, please refer to our development notes.

Copyright and license

Copyright 2020 D. Meyer and T. Nagler. Licensed under MIT.

Acknowledgements

Special thanks to @letmaik for his suggestions and contributions to the project.