Skip to main content

Generate realistic synthetic datasets with a simple and powerful API.

Project description

๐Ÿš€ DataForge

Generate realistic synthetic datasets with a simple and powerful API.

DataForge is a modern Python library for generating realistic synthetic datasets for testing, software development, machine learning, education, demonstrations, and AI projects.


โœจ Features

  • ๐Ÿš€ Generate thousands of realistic records in seconds
  • ๐Ÿ“ฆ 15 built-in datasets
  • ๐Ÿ›  Create custom datasets using schemas
  • ๐Ÿ“Š Returns Pandas DataFrame
  • ๐Ÿ’พ Export to CSV
  • ๐Ÿ“„ Export to JSON
  • ๐Ÿ“— Export to Excel (.xlsx)
  • ๐Ÿ—„ Export to SQLite
  • โšก Fast and lightweight
  • ๐ŸŒ Cross-platform
  • ๐Ÿ Python 3.10+

๐Ÿ“ฆ Installation

Install from PyPI:

pip install dataforgeplus

Or install from source:

git clone https://github.com/Aatif05-it/DataForge.git

cd DataForge

pip install -e .

๐Ÿš€ Quick Start

Generate an Employee Dataset

from dataforge import forge

df = forge(
    dataset="employee",
    rows=10
)

print(df.head())

Example Output

Name Email Department Salary
John Smith john@email.com IT 72000
Emma Wilson emma@email.com HR 61000

๐Ÿ›  Custom Dataset

from dataforge import forge

df = forge(
    rows=100,
    schema={
        "name": "name",
        "email": "email",
        "phone": "phone",
        "salary": "salary"
    }
)

print(df.head())

๐Ÿ“š Available Datasets

from dataforge import available_datasets

print(available_datasets())

Output

[
'airline',
'banking',
'countries',
'creditcard',
'ecommerce',
'employee',
'hospital',
'hr',
'library',
'mobile',
'realestate',
'restaurant',
'student',
'vehicle',
'weather'
]

๐Ÿ’พ Export Examples

CSV

df.to_csv("employees.csv", index=False)

Excel

df.to_excel("employees.xlsx", index=False)

JSON

df.to_json(
    "employees.json",
    orient="records",
    indent=4
)

SQLite

import sqlite3

conn = sqlite3.connect("employees.db")

df.to_sql(
    "employees",
    conn,
    if_exists="replace",
    index=False
)

conn.close()

๐Ÿ“š Complete Example

from dataforge import forge

employees = forge(
    dataset="employee",
    rows=100
)

employees.to_csv("employees.csv", index=False)

print(employees.head())

๐Ÿท Built-in Datasets

  • Airline
  • Banking
  • Countries
  • Credit Card
  • Ecommerce
  • Employee
  • Hospital
  • HR
  • Library
  • Mobile
  • Real Estate
  • Restaurant
  • Student
  • Vehicle
  • Weather

๐Ÿ”ง Supported Providers

Some commonly used providers include:

name
first_name
last_name
email
phone
city
state
country
company
department
salary
gender
uuid
date
price
product
invoice_number
order_id
doctor
student_id
employee_id
vehicle
credit_card
passport_number
pan
aadhaar

๐Ÿ“ Project Structure

DataForge/
โ”‚
โ”œโ”€โ”€ dataforge/
โ”‚   โ”œโ”€โ”€ __init__.py
โ”‚   โ”œโ”€โ”€ forge.py
โ”‚   โ”œโ”€โ”€ cli.py
โ”‚   โ”œโ”€โ”€ datasets/
โ”‚   โ”œโ”€โ”€ providers/
โ”‚   โ””โ”€โ”€ utils/
โ”‚
โ”œโ”€โ”€ docs/
โ”œโ”€โ”€ examples/
โ”œโ”€โ”€ tests/
โ”œโ”€โ”€ LICENSE
โ”œโ”€โ”€ MANIFEST.in
โ”œโ”€โ”€ pyproject.toml
โ”œโ”€โ”€ setup.py
โ””โ”€โ”€ README.md

๐Ÿ—บ Roadmap

  • โœ… CSV Export
  • โœ… JSON Export
  • โœ… Excel Export
  • โœ… SQLite Export
  • ๐Ÿ”œ YAML Schema Support
  • ๐Ÿ”œ AI Dataset Generator
  • ๐Ÿ”œ PostgreSQL Export
  • ๐Ÿ”œ MongoDB Export
  • ๐Ÿ”œ Parquet Export
  • ๐Ÿ”œ REST API

๐Ÿ“‹ Requirements

  • Python 3.10+
  • pandas
  • Faker
  • openpyxl

๐Ÿ“œ License

This project is licensed under the MIT License.


๐Ÿ‘จโ€๐Ÿ’ป Author

Khan Aatif

GitHub

https://github.com/Aatif05-it


๐Ÿค Contributing

Contributions are welcome.

  1. Fork the repository.
  2. Create a feature branch.
  3. Commit your changes.
  4. Push to your fork.
  5. Open a Pull Request.

โญ Support

If you find DataForge useful, please consider giving the project a โญ on GitHub.

Repository:

https://github.com/Aatif05-it/DataForge


๐Ÿ“ง Contact

Email:

mohdaatif9820@gmail.com

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dataforgeplus-1.0.1.tar.gz (30.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dataforgeplus-1.0.1-py3-none-any.whl (56.6 kB view details)

Uploaded Python 3

File details

Details for the file dataforgeplus-1.0.1.tar.gz.

File metadata

  • Download URL: dataforgeplus-1.0.1.tar.gz
  • Upload date:
  • Size: 30.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.4

File hashes

Hashes for dataforgeplus-1.0.1.tar.gz
Algorithm Hash digest
SHA256 88e0ad5b6a717d1da869b35ea327ba7ce594ce8010372d3d6d9217141b52d89e
MD5 955da85cfd96df3d1f9d1a18f0c66322
BLAKE2b-256 f9d5926aeff4fb18d810d99296f1f4770175d4f0afbe8d90c80b6c2bb60142f0

See more details on using hashes here.

File details

Details for the file dataforgeplus-1.0.1-py3-none-any.whl.

File metadata

  • Download URL: dataforgeplus-1.0.1-py3-none-any.whl
  • Upload date:
  • Size: 56.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.4

File hashes

Hashes for dataforgeplus-1.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 bf3aa44bea8e68dce80573f61a6336ac9b01866f01602bdd6fc9ec7e5f9811f3
MD5 fba73cd80ccb6a304007daa0538374d9
BLAKE2b-256 f5eea85dc7bcab327730fecdb6689c2f0707e3e943fb882aa21eaf42d9f66eb7

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page