VD3

A DVC-backed media database for computer vision and ML developers. Tracks video and imageset assets, annotations, and worksets as MP4/JSON media with CSV-based metadata, so datasets stay versioned and reproducible across local disks and remote storage backends.

Contents

• Installation
• Quick Start
• Core Concepts
• Adding Assets
  • Videos
  • Imagesets
  • Annotation layers
  • Layer groups
  • COCO
  • Git-backed vs. DVC-backed annotations
  • Layer versioning (status, push, pull, revert)
• Worksets
• Remote Storage
• Listing & Inspection
• Exporting
• Library API
• Versioning & stability
  • Public API
  • Deprecations
• CLI Reference

Installation

pip install vd3

With uv run:

uv pip install vd3

Add as a dependency to your project:

# pyproject.toml
[project]
dependencies = ["vd3"]

Then resolve dependencies and sync:

uv sync

Quick Start

# Initialize a content database in the current directory
vd3 db init

# ...or in a specific directory
vd3 db init /path/to/mydb

# Add a video under a datasource
vd3 datasource add-video my-datasource clip.mp4 -p /path/to/mydb

# Add multiple videos with a glob (quote to prevent shell expansion)
vd3 datasource add-video my-datasource '*.mp4' -p /path/to/mydb

# List assets in a datasource
vd3 datasource assets my-datasource -p /path/to/mydb

# Show media availability
vd3 media status -p /path/to/mydb

Every command reads vd3 <noun> <verb> [args]. The nouns are db, datasource, workset, asset, layer, and media. Containers (datasource, workset) own ingest verbs and contained-asset listings; the asset noun is reserved for operations on a known asset.

All commands that accept --path also honor the VD3_DB environment variable, so you can point at a database once and drop the -p flag from individual commands. On startup vd3 automatically loads a .env file from the current directory or the nearest ancestor, so a project-level .env containing:

VD3_DB=/path/to/mydb

is enough — no export or shell sourcing required:

vd3 asset list
vd3 media status

Precedence: explicit --path > shell-exported VD3_DB > .env VD3_DB > current directory. Shell-exported values win over .env so you can do one-off overrides without editing the file.

--path on a specific command still overrides the env var.

Core Concepts

• Datasource — groups assets by origin (e.g. dashcam-2024, test-data). Required when importing.
• Asset — a single video or imageset (directory of images). A video is backed either by an MP4 (ffmpeg-decoded) or an uncompressed .tar of images (a "tar-of-images video" — frames served straight from the archive, kept intact as one DVC blob); import either with vd3 datasource add-video.
• Workset — a named subset of assets, optionally organized into packages (folders). Independent of storage layout.
• Annotation layer — detections or tracks attached to an asset, with a key (e.g. gt, det/yolo-v8) and a human or machine source. Layers are git-backed by default (small, always present after clone); large layers can be DVC-backed (fetched on demand like media) via add_annotation_layer(..., dvc=True) / vd3 layer add-vd3 --dvc, or converted later with vd3 layer migrate-to-dvc.
• Layer group — a per-layer identifier that groups the outputs of one computer-vision solution together (a CNN detector + a background-model MTI detector + the tracker built from either, all belonging to the same pipeline release). Downstream selectors (e.g. the visdata layer picker) show one entry per group and expose the individual layers as sub-toggles. Defaults to the trailing segment of the layer key after the last /; can be set explicitly. See Layer groups for examples.
• Per-image metadata — arbitrary non-bbox metadata attached per image (labels, attributes, …), stored as a DVC-backable image_metadata layer via add_image_metadata / read_image_metadata. Keeps bulky per-image metadata out of the git-tracked imageset.json manifest; vd3 layer migrate-imageset-extra moves existing inline extra into one.

Adding Assets

Videos

# Single file
vd3 datasource add-video dashcam clip.mp4

# Glob (recursive)
vd3 datasource add-video dashcam 'rawdata/**/*.mp4'

# Tar-of-images video (frames served from the archive; --fps stamps a nominal rate)
vd3 datasource add-video dashcam frames.tar --fps 30

# Force re-import of a duplicate (matched by SHA-256)
vd3 datasource add-video dashcam clip.mp4 --force

# Add and assign to a workset/package
vd3 datasource add-video dashcam clip.mp4 -w my-workset -k batch1

Imagesets

# Directory of images
vd3 datasource add-imageset my-datasource /path/to/images

# Tar archive
vd3 datasource add-imageset my-datasource images.tar

Annotation layers

VD3 JSON detections/tracks into an existing asset. The second positional is a layer-name prefix prepended to every layer key in the file (e.g. importing a file with det/yolo under run-1 produces run-1/det/yolo):

vd3 layer add-vd3 clip run-1 results.json -p /path/to/mydb

Layers can also be added directly from Python — pass the frame-indexed annotations plus a source (GROUND_TRUTH / HUMAN_ANNOTATED / RESULT) and a layer_type (detections / tracks):

from vd3storage import VD3Storage

with VD3Storage("/path/to/mydb") as db:
    asset = db.get_asset("dashcam", "clip1")
    db.add_annotation_layer(
        asset.asset_id,
        layer="det/cnn-v1",
        display_name="CNN detector v1",
        annotations={0: [{"class_name": "car", "bbox": [10, 20, 30, 40], "confidence": 0.91}]},
        source="RESULT",
        layer_type="detections",
    )

Layer groups

A layer_group labels each annotation layer with the computer-vision solution it belongs to, so multiple outputs from the same pipeline surface together in downstream tools. This matters as soon as a single "solution" writes more than one layer per asset — which is the common case for modern CV pipelines. Consider a solution named sb-v3 that runs three stages:

All three carry layer_group="sb-v3". When you open the asset in visdata, the layer picker shows one entry for sb-v3, and clicking it exposes the three underlying overlays individually — you can toggle the CNN detections, the MTI detections, and the resulting tracks against each other without hunting through the full flat list.

Default rule — when layer_group is not set explicitly, vd3 uses the trailing segment of the layer key after the last /, so a common naming convention already groups related layers together for free:

# All three land in group "sb-v3" via the default rule (no layer_group arg needed).
db.add_annotation_layer(asset.asset_id, "det/sb-v3",   "SB v3 CNN", cnn_dets, source="RESULT", layer_type="detections")
db.add_annotation_layer(asset.asset_id, "det/sb-v3-mti", "SB v3 MTI", mti_dets, source="RESULT", layer_type="detections")  # default -> "sb-v3-mti"
db.add_annotation_layer(asset.asset_id, "track/sb-v3", "SB v3 tracks", tracks, source="RESULT", layer_type="tracks")

Notice the MTI layer defaults to its own group because the trailing segment is sb-v3-mti. To pull it into the shared sb-v3 group, pass layer_group= explicitly:

db.add_annotation_layer(
    asset.asset_id,
    layer="det/sb-v3-cnn",
    display_name="SB v3 CNN",
    annotations=cnn_dets,
    source="RESULT",
    layer_type="detections",
    layer_group="sb-v3",          # <- explicit
)
db.add_annotation_layer(
    asset.asset_id,
    layer="det/sb-v3-mti",
    display_name="SB v3 MTI",
    annotations=mti_dets,
    source="RESULT",
    layer_type="detections",
    layer_group="sb-v3",          # <- share the group with the CNN layer
)
db.add_annotation_layer(
    asset.asset_id,
    layer="track/sb-v3",
    display_name="SB v3 tracks",
    annotations=tracker_output,
    source="RESULT",
    layer_type="tracks",
    layer_group="sb-v3",          # <- and with the tracker built from them
)

When you write a layer JSON directly — for example when generating results from a model-serving pipeline that emits VD3 JSON files — set the layer_group field in the body itself:

{
  "layer": "det/sb-v3-mti",
  "display_name": "SB v3 MTI detections",
  "layer_type": "detections",
  "layer_group": "sb-v3",
  "source": "RESULT",
  "frames": { "0": [ ... ], "1": [ ... ] }
}

Import it the same way as any VD3 JSON layer (vd3 layer add-vd3 …); the layer_group value round-trips into the asset_layers table on write and is re-read from the JSON body on future scans. The database-wide inventory shows one column per stored field:

vd3 layer list          # ↕ per layer_group aggregation (asset_layers table)
vd3 asset layers clip1  # per-asset detail

COCO

Import COCO annotations into an existing imageset:

vd3 layer add-coco my-imageset gt annotations.json \
    --source human --reviewed-all

Import a full COCO dataset (creates the imageset and imports annotations in one step):

vd3 datasource add-imageset-from-coco my-datasource gt annotations.json \
    --image-root /path/to/images

Git-backed vs. DVC-backed annotations

Annotation layers are git-backed by default — the JSON is committed directly, so it's always present after a clone (ideal for small ground-truth files). Large or bulky layers can instead be DVC-backed, like media: git tracks only a small .json.dvc pointer, and the body is fetched on demand and travels with its media on push/pull. (Media/video files are always DVC-backed; only annotations offer the choice.)

Add a new layer straight to DVC with --dvc:

vd3 layer add-vd3 clip run-1 results.json --dvc -p /path/to/mydb

Convert existing git-backed layers to DVC-backed in place — preview first, then scope by asset/layer or convert everything (optionally only layers above a size):

vd3 layer migrate-to-dvc --all --dry-run       # preview what would convert
vd3 layer migrate-to-dvc --all                 # convert every layer
vd3 layer migrate-to-dvc clip --layer det/qm   # or scope to one asset/layer
vd3 layer migrate-to-dvc --all --min-size 1    # only layers ≥ 1 MB

Migration git rm --cacheds each body, dvc adds it, and writes the .dvc pointer plus an annotations/.gitignore entry. Afterward, commit the new pointers and .gitignore, then push the bodies to your remote:

vd3 media push --all

migrate-to-dvc only acts on layers VD3 already tracks (created via vd3 layer … or the import commands). Raw JSON dropped into db/metadata/.../annotations/ by hand must be imported first (vd3 layer add-vd3 … --dvc, or import then migrate).

Layer versioning (status, push, pull, revert)

DVC-backed annotation layers have the same push/pull lifecycle as media, plus tooling to inspect sync state and undo local edits. These commands are grouped under a Versioning panel in vd3 layer --help. All of push / pull / revert scope with --asset / --datasource / --workset / --all.

See what's synced and what changed locally — a per-layer counterpart to vd3 media status:

vd3 layer status                     # summary + per-datasource breakdown
vd3 layer status --list              # one row per layer (Datasource/Asset/Layer/Backing/Status)
vd3 layer status --datasource dashcam

Each layer reports one of synced (on the remote), local (not pushed), remote-only (pointer present, body not pulled), modified (edited since its last dvc add, or uncommitted git changes for a git-backed layer), local (DVC-backed but no remote configured), or git-tracked.

Push / pull layer bodies — a layer-only counterpart to vd3 media push / pull (which bundle media and layers). Use these to move just the annotations:

vd3 layer push --datasource dashcam  # upload DVC-backed layers in the datasource
vd3 layer pull --workset my-experiment
vd3 layer push --all                 # every DVC-backed layer in the database

A layer shown as modified won't upload its new bytes with a plain push — dvc push sends what the .dvc pointer currently references. Re-record it first (dvc add db/metadata/.../annotations/<layer>.json), then vd3 layer push.

Revert local modifications — discard edits and restore the last-recorded content (DVC-backed bodies from the DVC cache, git-backed tracked bodies from git). Dry-run by default, so you always preview first:

vd3 layer revert --all               # DRY RUN — lists what would be reverted, changes nothing
vd3 layer revert --all --no-dry-run  # actually restore
vd3 layer revert --asset clip        # scope to one asset

Untracked git-backed bodies have no previously-recorded version, so revert leaves them untouched.

Worksets

# Create
vd3 workset create "My Experiment"

# Add assets by name or ID
vd3 workset add my-experiment clip-001 clip-002

# ...or by media-path glob (run from the database root; files must be on disk)
cd /path/to/mydb
vd3 workset add my-experiment 'db/media/videos/fc/*.mp4'

# Inspect
vd3 workset list
vd3 workset show my-experiment      # metadata + layers + packages
vd3 workset assets my-experiment    # assets in the workset

# Remove an asset / delete the workset
vd3 workset remove my-experiment clip-001
vd3 workset delete my-experiment

Remote Storage

Media files are tracked by DVC. A content database has a single configured remote.

# Set the remote (replaces any existing one)
vd3 media remote set gs://my-bucket/vd3-data
vd3 media remote show

# Sync (push and pull both accept --workset/-w, --asset/-a, --datasource/-d, --all)
vd3 media push --all
vd3 media pull --workset my-experiment
vd3 media status

Supported backends:

Listing & Inspection

vd3 asset list                       # all assets (cross-container)
vd3 datasource list                  # all datasources
vd3 datasource assets dashcam        # assets in a datasource
vd3 datasource assets dashcam --paths      # one media path per line
vd3 datasource assets dashcam --filenames  # one filename per line
vd3 datasource layers dashcam        # annotation layers across the datasource
vd3 workset assets my-experiment     # assets in a workset
vd3 workset layers my-experiment     # annotation layers across the workset
vd3 asset layers clip                # annotation layers on an asset
vd3 layer list                       # annotation layers across the whole database (per-layer coverage)
vd3 layer status                     # per-layer DVC sync state + local modifications
vd3 asset show clip                  # asset details
vd3 db info                          # database overview
vd3 db query "SELECT ..."            # raw DuckDB SQL against the CSV tables

Exporting

# Extract frames from a video or images from an imageset
vd3 asset export-frames clip -o ./out

Library API

The CLI is a thin wrapper around VD3Storage, which is also usable directly.

from vd3storage import VD3Storage, Asset, Workset  # Tag, WorksetAsset also exported

# Open an existing database (or use VD3Storage.init(path) to create one)
storage = VD3Storage("/path/to/mydb")

# Browse assets
for a in storage.list_assets(datasource="dashcam"):
    print(f"{a.name} ({a.asset_type}): {a.frame_count} frames @ {a.nominal_fps} fps")

# Look up by (datasource, name) or by ID
clip = storage.get_asset("dashcam", "clip-001")
clip = storage.get_asset_by_id("3f1a...")

# Import a video
asset = storage.import_video("clip.mp4", datasource="dashcam")

# Resolve where the media file lives on disk
storage.resolve_media_path(clip)

# Annotation layers
storage.list_annotation_layers(clip.asset_id)
storage.read_annotation_layer(clip.asset_id, "gt")

# Worksets
ws = storage.create_workset("My Experiment")
storage.add_asset_to_workset(ws.workset_id, clip.asset_id, package="batch1")
storage.list_workset_assets(ws.workset_id)

# Raw DuckDB SQL against the underlying CSV tables
rows = storage.execute_sql("SELECT name, frame_count FROM assets WHERE asset_type = 'video'")

Other useful methods: import_imageset, import_coco, import_coco_dataset, import_result, export_coco, open_video, open_imageset, get_frame_image, add_tag, is_media_available, pull, push. Inspect help(VD3Storage) for the full surface.

Versioning & stability

The package follows Semantic Versioning. All notable changes are recorded in CHANGELOG.md, and every release is tagged vX.Y.Z in git.

While the package is in 0.x, minor bumps (0.2 → 0.3) may contain breaking changes; patch bumps (0.2.1 → 0.2.2) are backwards compatible. Every breaking change is called out under a ### Breaking heading in the CHANGELOG entry for that release. Once the package reaches 1.0 it will follow strict SemVer (MAJOR = breaking, MINOR = additive, PATCH = fix).

Public API

A change is "breaking" only if it alters one of the following:

1. Names re-exported from the top-level vd3storage package (i.e. listed in vd3storage.__all__):
   • VD3Storage and its documented methods
   • AlreadyInitializedError
   • The model classes Asset, Evaluation, Tag, Workset, WorksetAsset (including their field names and types)
   • __version__
2. The vd3 CLI — command names, option names, exit codes, and the documented input file formats (VD3 JSON, COCO).
3. The on-disk layout of a content database — directory structure under db/, CSV table schemas (tracked by SCHEMA_VERSION in db/tables/), the shape of video.json / imageset.json / annotation JSON files, and the structure of generated pyproject.toml / .gitignore.

Everything else is internal and may change without a major-version bump even if it is reachable via an import path. That includes the vd3storage.orm, vd3storage.dvc, vd3storage.media, vd3storage.importers, vd3storage.exporters, and vd3storage.cli submodules; helper functions in vd3storage.storage that start with _; and the on-disk format of files written into .dvc/ (those belong to DVC).

Deprecations

When a public API needs to change incompatibly, the old form keeps working and emits DeprecationWarning for at least one minor release before being removed. Current deprecations are listed in the CHANGELOG under each release's ### Deprecated heading.

To surface them in your own code:

python -W "default::DeprecationWarning:vd3storage" your_script.py

CLI Reference

vd3 --help                       Top-level help
vd3 <noun> --help                Help for a noun
vd3 <noun> <verb> --help         Help for a specific command

Every command reads vd3 <noun> <verb> [args]. Positionals carry identity (target → composite parts → payload); flags carry modifiers (--paths, --filenames, --source human, ...).