Skip to main content

General framework for synthetic data generation

Project description

🎨 NeMo Data Designer

CI License Python 3.10 - 3.13 NeMo Microservices Code Tokens

Generate high-quality synthetic datasets from scratch or using your own seed data.


Welcome!

Data Designer helps you create synthetic datasets that go beyond simple LLM prompting. Whether you need diverse statistical distributions, meaningful correlations between fields, or validated high-quality outputs, Data Designer provides a flexible framework for building production-grade synthetic data.

What can you do with Data Designer?

  • Generate diverse data using statistical samplers, LLMs, or existing seed datasets
  • Control relationships between fields with dependency-aware generation
  • Validate quality with built-in Python, SQL, and custom local and remote validators
  • Score outputs using LLM-as-a-judge for quality assessment
  • Iterate quickly with preview mode before full-scale generation

⚠️ Security Notice: LiteLLM Supply-Chain Incident (2026-03-24)

On March 24, 2026, malicious versions of litellm (1.82.7 and 1.82.8) were published to PyPI containing a credential stealer. The compromised packages were available for approximately five hours (10:39 – 16:00 UTC) before being removed.

The only Data Designer releases that could resolve to these versions are v0.2.2 (Dec 2025) and v0.2.3 (Jan 2026), which carried a looser litellm<2 upper bound. These are nearly three months old and have been superseded by eight subsequent releases — both have been yanked from PyPI as a precaution. All other releases (v0.3.0 – v0.5.3) pinned litellm to >=1.73.6,<1.80.12 and were never compatible with 1.82.x. Starting with v0.5.4, litellm is no longer a dependency.

To have been impacted through Data Designer, you would need to have had one of these two old versions explicitly pinned and run a fresh pip install or dependency-cache update that resolved litellm during the five-hour window on March 24. If you believe you may be affected, see BerriAI's incident report for remediation steps.


Quick Start

1. Install

pip install data-designer

Or install from source:

git clone https://github.com/NVIDIA-NeMo/DataDesigner.git
cd DataDesigner
make install

2. Set your API key

Start with one of our default model providers:

Grab your API key(s) using the above links and set one or more of the following environment variables:

export NVIDIA_API_KEY="your-api-key-here"

export OPENAI_API_KEY="your-openai-api-key-here"

export OPENROUTER_API_KEY="your-openrouter-api-key-here"

3. Start generating data!

import data_designer.config as dd
from data_designer.interface import DataDesigner

# Initialize with default settings
data_designer = DataDesigner()
config_builder = dd.DataDesignerConfigBuilder()

# Add a product category
config_builder.add_column(
    dd.SamplerColumnConfig(
        name="product_category",
        sampler_type=dd.SamplerType.CATEGORY,
        params=dd.CategorySamplerParams(
            values=["Electronics", "Clothing", "Home & Kitchen", "Books"],
        ),
    )
)

# Generate personalized customer reviews
config_builder.add_column(
    dd.LLMTextColumnConfig(
        name="review",
        model_alias="nvidia-text",
        prompt="Write a brief product review for a {{ product_category }} item you recently purchased.",
    )
)

# Preview your dataset
preview = data_designer.preview(config_builder=config_builder)
preview.display_sample_record()

What's next?

📚 Learn more

🔧 Configure models via CLI

data-designer config providers # Configure model providers
data-designer config models    # Set up your model configurations
data-designer config list      # View current settings

🤖 Agent Skill

Data Designer has a skill for coding agents. Just describe the dataset you want, and your agent handles schema design, validation, and generation. While the skill should work with other coding agents that support skills, our development and testing has focused on Claude Code at this stage.

Install via skills.sh (be sure to select Claude Code as an additional agent):

npx skills add NVIDIA-NeMo/DataDesigner

After installation, type /data-designer or describe the dataset you want and the skill will kick in.

🤝 Get involved

This repository supports agent-assisted development — see CONTRIBUTING.md for the recommended workflow.


Telemetry

Data Designer collects telemetry to help us improve the library for developers. We collect:

  • The names of models used
  • The count of input tokens
  • The count of output tokens

No user or device information is collected. This data is not used to track any individual user behavior. It is used to see an aggregation of which models are the most popular for SDG. We will share this usage data with the community.

Specifically, a model name that is defined a ModelConfig object, is what will be collected. In the below example config:

ModelConfig(
    alias="nv-reasoning",
    model="openai/gpt-oss-20b",
    provider="nvidia",
    inference_parameters=ChatCompletionInferenceParams(
        temperature=0.3,
        top_p=0.9,
        max_tokens=4096,
    ),
)

The value openai/gpt-oss-20b would be collected.

To disable telemetry capture, set NEMO_TELEMETRY_ENABLED=false.

Top Models

This chart represents the breakdown of models used for Data Designer across all synthetic data generation jobs from 2/23/2026 to 3/23/2026.

Top models used for synthetic data generation

Last updated on 3/23/2026


License

Apache License 2.0 – see LICENSE for details.


Citation

If you use NeMo Data Designer in your research, please cite it using the following BibTeX entry:

@misc{nemo-data-designer,
  author = {The NeMo Data Designer Team, NVIDIA},
  title = {NeMo Data Designer: A framework for generating synthetic data from scratch or based on your own seed data},
  howpublished = {\url{https://github.com/NVIDIA-NeMo/DataDesigner}},
  year = {2025},
  note = {GitHub Repository},
}

Project details


Release history Release notifications | RSS feed

This version

0.5.6

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

data_designer-0.5.6.tar.gz (119.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

data_designer-0.5.6-py3-none-any.whl (99.0 kB view details)

Uploaded Python 3

File details

Details for the file data_designer-0.5.6.tar.gz.

File metadata

  • Download URL: data_designer-0.5.6.tar.gz
  • Upload date:
  • Size: 119.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.9

File hashes

Hashes for data_designer-0.5.6.tar.gz
Algorithm Hash digest
SHA256 5e56ce7f5356470d3087972a8fd9245d20cf8ebe4c2d6007f5c7780b19e3e179
MD5 09967f324081429f474d432bb3268b0c
BLAKE2b-256 03739f1ce61b9cf2a65ac7eaef79d5fcd2a7ea27eed0590c5ab6188bc874aa43

See more details on using hashes here.

File details

Details for the file data_designer-0.5.6-py3-none-any.whl.

File metadata

  • Download URL: data_designer-0.5.6-py3-none-any.whl
  • Upload date:
  • Size: 99.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.9

File hashes

Hashes for data_designer-0.5.6-py3-none-any.whl
Algorithm Hash digest
SHA256 5277ec6ccf58538b30cd6582b38d5caae4c68063917ac7a9d60daa8e7a157558
MD5 156f2b6502c04af3c9c373942d5c6cc0
BLAKE2b-256 f1d611bb554853e1fdb80f62095fb6f7bfdd5c053633249f1cb1be721f186f7d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page