Skip to main content

A synthetic data generation engine

Project description

Gener8

Gener8 is a Python-based synthetic data generation engine using neural networks. It loads data, trains a GAN-based model, and generates synthetic data that mimics the input's structure, including missing values.


Features

  • Data Connector: Load data from CSV, JSON, Excel, or pandas DataFrames.
  • Trainer: Train a Gaussian Mixture Model to capture data distributions.
  • Generator: Produce synthetic data with numerical and categorical columns.
  • Modular Design: Each component is independent and reusable.
  • Pip Installable: Easily install and integrate into your projects.

Installation

To install Gener8, use pip:

pip install gener8

Alternatively, to install from source:

# Clone the repository
git clone https://github.com/abdulrahman0044/gener8.git
cd gener8

# Install dependencies
pip install -r requirements.txt

# Install the package
pip install .

Usage

Below is an example of how to use Gener8 to generate synthetic data:

import pandas as pd
import numpy as np
from gener8 import Gener8Engine

# Initialize the engine
engine = Gener8Engine()

# Create sample data
data = pd.DataFrame({
    'age': np.random.normal(30, 10, 1000),
    'income': np.random.normal(50000, 10000, 1000),
    'category': np.random.choice(['A', 'B', 'C'], 1000)
})

# Load and train the model
engine.load_and_train(
    data,
    n_components=3,
    epochs=5,
    max_sample_size=10000,
    batch_size=32,
    gradient_accumulation_steps=2,
    max_training_time=3600,
    max_sequence_window=1,
    enable_flexible_generation=True,
    value_protection=True,
    rare_category_replacement_method="CONSTANT",
    differential_privacy=None)

# Generate synthetic data
synthetic_data = engine.generate(100)
print(synthetic_data.head())

Requirements

  • Python >= 3.8
  • pandas >= 1.5.0
  • numpy >= 1.23.0
  • scikit-learn >= 1.1.0

Project Structure

gener8/
├── gener8/
│   ├── __init__.py
│   ├── data_connector.py
│   ├── trainer.py
│   ├── generator.py
│   ├── engine.py
│   └── evaluation.py
├── setup.py
├── README.md
├── requirements.txt
├── LICENSE
├── MANIFEST.in

Development

To contribute or modify Gener8:

# Clone the repository
git clone https://github.com/abdulrahman0044/gener8.git

# Install dependencies
pip install -r requirements.txt

# Make changes and test locally

# Build the package
python -m build

# Install locally
pip install dist/gener8-0.1.0-py3-none-any.whl

License

This project is licensed under the Apache 2.0 License. See the LICENSE file for details.


Contact

For questions or contributions, please open a pull request or a issue on the GitHub repository or contact abdulrahamanbabatunde12@gmail.com.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gener8_synth-0.1.0.tar.gz (15.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

gener8_synth-0.1.0-py3-none-any.whl (16.1 kB view details)

Uploaded Python 3

File details

Details for the file gener8_synth-0.1.0.tar.gz.

File metadata

  • Download URL: gener8_synth-0.1.0.tar.gz
  • Upload date:
  • Size: 15.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.11.1

File hashes

Hashes for gener8_synth-0.1.0.tar.gz
Algorithm Hash digest
SHA256 708027e973d0ae6d7d113f33a7328d6c5a048ce29f42a62811941e0735f20703
MD5 7d959f7d679b130a191fd7156abf66ac
BLAKE2b-256 d103240a7e7e45ed5cacc416f6275567d0b95e3af0395c27be33123f2cb29ce5

See more details on using hashes here.

File details

Details for the file gener8_synth-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: gener8_synth-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 16.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.11.1

File hashes

Hashes for gener8_synth-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 7a3515ffbc499fd51b8d23db762c8cd02efcb0b2000568a6e90b19d73d0c9690
MD5 5ac55a3c2d95d7cae7a5db0da006939b
BLAKE2b-256 d4a8b7f2b11fd1e7888336ef2a048cf9ed68a9846fccd834139559e7ae0968cd

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page