data-designer 0.1.0

General framework for synthetic data generation

🎨 NeMo Data Designer

Generate high-quality synthetic datasets from scratch or using your own seed data.

---

Welcome!

Data Designer helps you create synthetic datasets that go beyond simple LLM prompting. Whether you need diverse statistical distributions, meaningful correlations between fields, or validated high-quality outputs, Data Designer provides a flexible framework for building production-grade synthetic data.

What can you do with Data Designer?

• Generate diverse data using statistical samplers, LLMs, or existing seed datasets
• Control relationships between fields with dependency-aware generation
• Validate quality with built-in Python, SQL, and custom local and remote validators
• Score outputs using LLM-as-a-judge for quality assessment
• Iterate quickly with preview mode before full-scale generation

---

Quick Start

1. Install

pip install data-designer

Or install from source:

git clone https://github.com/NVIDIA-NeMo/DataDesigner.git
cd DataDesigner
make install

2. Set your API key

Get your API key from build.nvidia.com or OpenAI:

export NVIDIA_API_KEY="your-api-key-here"
# Or use OpenAI
export OPENAI_API_KEY="your-openai-api-key-here"

3. Generate your first dataset

from data_designer.essentials import (
    CategorySamplerParams,
    DataDesigner,
    DataDesignerConfigBuilder,
    LLMTextColumnConfig,
    PersonSamplerParams,
    SamplerColumnConfig,
    SamplerType,
)

# Initialize with default settings
data_designer = DataDesigner()
config_builder = DataDesignerConfigBuilder()

# Add a product category
config_builder.add_column(
    SamplerColumnConfig(
        name="product_category",
        sampler_type=SamplerType.CATEGORY,
        params=CategorySamplerParams(
            values=["Electronics", "Clothing", "Home & Kitchen", "Books"],
        ),
    )
)

# Generate personalized customer reviews
config_builder.add_column(
    LLMTextColumnConfig(
        name="review",
        model_alias="nvidia-text",
        prompt="""Write a brief product review for a {{ product_category }} item you recently purchased.""",
    )
)

# Preview your dataset
preview = data_designer.preview(config_builder=config_builder)
preview.display_sample_record()

That's it! You've created a dataset.

---

What's next?

📚 Learn more

• Quick Start Guide – Detailed walkthrough with more examples
• Tutorial Notebooks – Step-by-step interactive tutorials
• Column Types – Explore samplers, LLM columns, validators, and more
• Model Configuration – Configure custom models and providers

🔧 Configure models via CLI

data-designer config providers # Configure model providers
data-designer config models    # Set up your model configurations
data-designer config list      # View current settings

🤝 Get involved

• Contributing Guide – Help improve Data Designer
• GitHub Issues – Report bugs or request features
• GitHub Discussions – Ask questions and share ideas

---

License

Apache License 2.0 – see LICENSE for details.