NVIDIA TAO DAFT: Dataset Annotation Format Toolkit

A toolkit for vision-language dataset formats — JSON-schema specs plus CLI/Python tools to validate and convert between them.

Overview

VLM dataset workflows have a contract-drift problem. Annotation pipelines emit data in one shape, training pipelines expect another, and the glue between them is ad-hoc adapter code that silently goes stale. Field renames, new optional values, schema-vs-data mismatches — these surface as training-time bugs rather than at the producer / consumer boundary where they belong.

What DAFT is:

• Schemas for vision-language dataset shapes — both annotation (what producers emit) and training (what consumers expect).
• A CLI + validator so anyone holding a dataset can check it against its schema before handing it off.
• Converters between annotation and training shapes — explicit, deterministic, with optional flags for media handling.
• A reference Python adapter that plugs one of the training shapes into cosmos-rl SFT.

New formats, validators, converters, and adapters are welcome; the same registration pattern that wires the built-ins works for your own extensions.

Value, by audience:

Quick start

# Install (direct from git)
pip install git+https://gitlab-master.nvidia.com/nvidia-tao-toolkit/experimental/nvidia-tao-daft.git

# Install (from wheel)
pip install nvidia-tao-daft

# Verify
tao-daft --help

For runnable examples, see examples/ and the CLI reference.

Documentation

Repository structure

nvidia-tao-daft/
├── examples/datasets/        # Working datasets, one subdir per format
│
├── tests/                    # Test suite (schemas, validators, converters, CLI, doc consistency)
│
└── src/nvidia_tao_daft/
    ├── cli/                  # tao-daft entry point (validate, convert)
    ├── formats/              # Format specifications + JSON schemas
    ├── validators/           # Validation engine
    ├── converters/           # Format converters (pairs/)
    └── datasets/             # Training-loop adapters

Requirements

Python 3.10 – 3.13. Runtime dependencies: jsonschema, pydantic. Dev dependencies: see pyproject.toml.

Contributing

See CONTRIBUTING.md for the DCO sign-off requirement.

License

Apache 2.0.