mockmyschema 1.0.0

Generate realistic mock data from YAML schema definitions

MockMySchema

🎯 Generate realistic mock data from YAML schema definitions at scale

MockMySchema is a powerful Python CLI tool that transforms simple YAML schema definitions into realistic CSV datasets with proper foreign key relationships, unique constraints, and statistical distributions. Built for developers who need realistic test data that scales to millions of rows.

✨ One-Liner Demo

# Generate 1M customers + 5M orders with realistic relationships
mockmyschema generate ecommerce.yaml -o ./data --seed 42

🚀 Quick Start

Installation

pip install mockmyschema

Create Your First Schema

# Generate a template
mockmyschema create-template simple -o my_schema.yaml

# Edit the schema (or use as-is)
# Generate data
mockmyschema generate my_schema.yaml -o ./output

Example Schema

version: "1.0"
locale: en_US

tables:
  customers:
    rows: 100_000
    columns:
      customer_id:
        type: uuid
        primary_key: true
      name:
        type: name
      email:
        type: email
        unique: true
      tier:
        type: enum
        values: [bronze, silver, gold, platinum]
        weights: [50, 30, 15, 5]
      signup_date:
        type: datetime
        start: "2023-01-01"
        end: "2024-12-31"

  orders:
    rows: 500_000
    columns:
      order_id:
        type: sequence
        primary_key: true
      customer_id:
        type: ref
        table: customers
        column: customer_id
        distribution: zipf
      order_date:
        type: datetime
        after: customers.signup_date
        end: "2024-12-31"
      total:
        type: decimal
        min_value: 10.00
        max_value: 5000.00
        precision: 10
        scale: 2
        distribution: lognormal

Generated Output

# CSV (default)
mockmyschema generate schema.yaml -o ./data

# SQL INSERT statements
mockmyschema generate schema.yaml -o ./data --format sql

# Both CSV + SQL
mockmyschema generate schema.yaml -o ./data --format both

output/
├── customers.csv    # 100K realistic customers with emails, names, tiers
├── customers.sql    # SQL INSERT statements (batched, 1000 rows per INSERT)
├── orders.csv       # 500K orders with valid foreign keys and temporal ordering
└── orders.sql       # Ready to run in any SQL database

SQL output example:

-- Generated by MockMySchema
-- Table: customers (5 rows)

INSERT INTO customers (id, name, email, age) VALUES
(1, 'Allison Hill', '[e-mail]', 22),
(2, 'Noah Rhodes', '[e-mail]', 55),
(3, 'Angie Henderson', '[e-mail]', 49),
(4, 'Daniel Wagner', '[e-mail]', 39),
(5, 'Cristian Santos', '[e-mail]', 38);

🎪 Key Features

🔗 Smart Relationships

• Foreign Keys: Automatic reference pools with distribution control (uniform, zipf, normal)
• Temporal Ordering: after constraints ensure logical time sequences
• Referential Integrity: All foreign keys point to valid primary keys

📊 Statistical Distributions

• Uniform: Equal probability for all values
• Normal: Bell curve distribution with mean/std
• Log-normal: For realistic price, income, size data
• Zipf: Power law for popularity, frequency data
• Exponential: For time intervals, queue lengths

🌍 Realistic Data Types

• Primitive: sequence, uuid, int, float, decimal, string, bool, enum
• Semantic: name, email, phone, address, city, company (Faker-powered)
• Temporal: datetime, date with range and ordering constraints
• Reference: ref for foreign key relationships

⚡ Production Ready

• Memory Efficient: Chunked generation for millions of rows
• Deterministic: Seed support for reproducible datasets
• Fast: Numpy-vectorized generation
• Scalable: Handles complex schemas with deep dependencies

🎨 Developer Experience

• YAML First: Clean, readable schema definitions
• CLI Focused: Simple commands, rich output
• Template Library: Pre-built schemas for common domains
• Validation: Comprehensive schema validation with helpful errors

📋 Column Types Reference

Primitive Types

Type	Description	Parameters
sequence	Auto-increment integers	start, step
uuid	UUID v4 strings	None
int	Random integers	min_value, max_value, distribution
float	Random floats	min_value, max_value, precision, distribution
decimal	High-precision decimals	min_value, max_value, precision, scale, distribution
string	Random strings	min_length, max_length, prefix, suffix
bool	Boolean values	true_pct
enum	Enumerated values	values, weights

Semantic Types (Faker-powered)

Type	Description	Locale Support
name	Full names	✅
first_name	First names	✅
last_name	Last names	✅
email	Email addresses	✅
phone	Phone numbers	✅
address	Street addresses	✅
city	City names	✅
country	Country names	✅
company	Company names	✅
text	Lorem ipsum text	✅

Temporal Types

Type	Description	Parameters
datetime	Date and time	start, end, after, distribution
date	Date only	start, end, after, distribution

Reference Types

Type	Description	Parameters
ref	Foreign key reference	table, column, distribution

🎯 Distribution Types

• uniform: Equal probability (default)
• normal: Bell curve (specify mean, std)
• lognormal: Right-skewed for prices, sizes
• zipf: Power law for popularity rankings
• exponential: For time intervals, queue lengths

📚 Examples & Templates

Built-in Templates

# E-commerce with customers, products, orders
mockmyschema create-template ecommerce -o ecommerce.yaml

# Banking with accounts, transactions
mockmyschema create-template banking -o banking.yaml

# SaaS with organizations, users, projects
mockmyschema create-template saas -o saas.yaml

# Simple blog with users, posts
mockmyschema create-template simple -o blog.yaml

Real-World Examples

# Generate 10M row e-commerce dataset
mockmyschema generate ecommerce.yaml -o ./big_data --chunk-size 50000

# Compressed output for storage efficiency
mockmyschema generate banking.yaml -o ./bank_data --compress

# Reproducible datasets with seeds
mockmyschema generate saas.yaml -o ./test_data --seed 12345

# Validation without generation
mockmyschema validate my_schema.yaml

🏁 CLI Commands

Generate Data

mockmyschema generate schema.yaml [OPTIONS]

Options:
  -o, --output DIR          Output directory (default: ./output)
  --format [csv|sql|both]  Output format (default: csv)
  --seed INT               Random seed for reproducible generation
  --chunk-size INT         Chunk size for memory-efficient generation
  --compress              Compress output files with gzip
  --quiet                 Suppress progress output
  --validate-only         Only validate schema without generating data
  --stats                 Show generation statistics

Validate Schema

mockmyschema validate schema.yaml

Create Templates

mockmyschema create-template {simple,ecommerce,banking,saas} -o output.yaml

System Info

mockmyschema info

🆚 MockMySchema vs Alternatives

Feature	MockMySchema	Faker	Mockaroo
Schema-driven	✅ YAML	❌ Code only	✅ Web UI
Foreign Keys	✅ Smart pools	❌ Manual	✅ Limited
Distributions	✅ 7 types	❌ Limited	✅ Some
Scale	✅ Millions	❌ Memory bound	✅ Paid tiers
Temporal Logic	✅ after constraints	❌ None	❌ None
Reproducible	✅ Seed support	✅ Basic	❌ No
CLI First	✅ Rich CLI	❌ Library only	❌ Web only
Open Source	✅ MIT	✅ MIT	❌ Freemium

🛠 Advanced Usage

Complex Relationships

# Multi-level dependencies with temporal ordering
users → accounts → transactions
  ↓       ↓           ↓
signup  opened     after_opened

Distribution Examples

# Realistic price distribution (most items cheap, few expensive)
price:
  type: decimal
  min_value: 5.99
  max_value: 999.99
  distribution: lognormal
  
# Popular items get more orders (80/20 rule)
product_id:
  type: ref
  table: products
  column: product_id
  distribution: zipf
  
# Normal age distribution
age:
  type: int
  min_value: 18
  max_value: 80
  distribution: normal

Memory Optimization

# For large tables, use smaller chunks
large_table:
  rows: 10_000_000
  chunk_size: 100_000  # Process in 100K chunks

🗺 Roadmap

• 🔄 Multiple Output Formats: Parquet, JSON, Delta Lake
• 🚀 Parallel Generation: Multi-core processing for massive datasets
• 🌊 Streaming Output: Kafka, database connectors
• 🌐 Web UI: Visual schema builder and preview
• 📊 Data Profiling: Statistics and quality metrics
• 🔌 Plugin System: Custom generators and formats
• ☁️ Cloud Integration: S3, BigQuery, Snowflake outputs

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

Development Setup

git clone https://github.com/radha9887/mockmyschema.git
cd mockmyschema
pip install -e ".[dev]"
pytest

📄 License

MIT License - see LICENSE file for details.

🙏 Acknowledgments

• Faker: For the excellent semantic data generation library
• Click: For the beautiful CLI framework
• NumPy: For fast numerical computing
• PyYAML: For clean configuration parsing

---

Made with ❤️ for developers who need realistic test data

⭐ Star us on GitHub if MockMySchema helps your project!