Skip to main content

data-annotations

PyPI Documentation License CI

data-annotations is a Python package for attaching provenance and structured descriptions to the files and directories your workflows produce.

It writes plain JSON annotation sidecars that are easy to inspect, archive, and publish with research outputs:

  • files use artifact.ext.annotation.json
  • directories use data-annotations.json at their root

Optional Markdown README sidecars can be generated for human-readable summaries.

Automatic discovery uses only these current filenames. Explicit Python paths and the CLI --manifest option can load canonical schema 9 content under any filename; legacy filenames do not make legacy content compatible.

Documentation

The full documentation is organized as a Diátaxis site.

Other links:

Version 5.0.0 is the first stable public release. Releases through 4.1.1 were unsupported beta snapshots; see versioning and compatibility for the SemVer guarantees and the permanent schema 9 archival baseline.

The next release writes annotation_version: "10", which adds execution-aware provenance. Tools older than 6.0.0 reject those documents; see versioning and compatibility for the migration and the --annotation-version 9 escape hatch.

Installation

Install the core library from PyPI:

pip install data-annotations

Or add it to a project with uv:

uv add data-annotations

Install CLI support when you want the data-annotations command:

pip install "data-annotations[cli]"
uv add "data-annotations[cli]"

Quick start

Decorate a function that writes an artifact. When the function runs, data-annotations records provenance and writes the JSON sidecar.

from pathlib import Path

from data_annotations.annotations import record_file_annotation
from data_annotations.description import FieldDefinition


@record_file_annotation(
    title="Participant Cohort",
    summary="Participant-level cohort assignments.",
    fields=[
        FieldDefinition(
            name="participant_id",
            data_type="string",
            summary="Stable participant identifier.",
            required=True,
            nullable=False,
        ),
    ],
    primary_key=["participant_id"],
    artifact_kind="dataset",
    write_readme=True,
)
def write_participants(artifact_path: Path, input_path: Path) -> Path:
    participant_ids = [
        line.strip()
        for line in input_path.read_text(encoding="utf-8").splitlines()[1:]
        if line.strip()
    ]
    artifact_path.parent.mkdir(parents=True, exist_ok=True)
    artifact_path.write_text(
        "participant_id\n" + "\n".join(participant_ids) + "\n",
        encoding="utf-8",
    )
    return artifact_path


artifact_path = Path("outputs") / "participants.csv"
write_participants(
    artifact_path=artifact_path,
    input_path=Path("data/raw/participants.csv"),
)

This writes:

outputs/participants.csv
outputs/participants.csv.annotation.json
outputs/participants.csv.README.md

CLI

The CLI supports running annotated commands, retrospective annotation, provenance inspection, source recovery, and sanitized publish bundles.

data-annotations run --answers summary.yaml -- Rscript summarize.R raw.csv summary.csv
data-annotations annotate file path/to/participants.csv --write-readme
data-annotations annotate directory path/to/run-001 --recursive
data-annotations annotate tui
data-annotations provenance match path/to/participants.csv
data-annotations provenance chain path/to/participants.csv
data-annotations provenance checkout path/to/participants.csv
data-annotations publish path/to/run-001 path/to/publish-bundle

Development

From a source checkout (assuming you have Task installed):

task install
task lint
task type-check
task test

Build or preview the documentation site:

task docs-build
task docs-serve

TUI tests

TUI tests use the shared readiness helpers in tests/tui/helpers.py to wait for the selected form's target, focus, and layout before asserting UI state. The waits process messages and refreshes, polling every 50 ms with a five-second deadline that reports the expected condition and current focus. Use these helpers instead of assuming that one pilot.pause() completes a selection or layout update.

Python runtime upgrades

The local .python-version pin selects Python 3.14. CI sets UV_PYTHON from each job's PYTHON_VERSION so uv uses the requested matrix version instead of the local pin. After synchronization, each job prints its interpreter version and executable and checks that the version matches the job configuration.

Renovate groups updates to .python-version and the CI PYTHON_VERSION variable. Upgrades to a new Python release line, such as 3.14 to 3.15, require approval in Renovate's Dependency Dashboard before it creates a branch or merge request. These upgrades use a separate python-runtime upgrades group; patch updates within a release line retain the python-runtime group and do not require dashboard approval.

Before approving a new release line, check dependency compatibility and wheel availability for the CI image. When adopting it, add the new Python version to the CI test matrix and update the supported Python classifiers in pyproject.toml.

Metadata

Release files for data-annotations 6.0.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for data-annotations 6.0.1
File Size Uploaded Provenance Actions
data_annotations-6.0.1.tar.gz 105.7 kB Without provenance Details for data_annotations-6.0.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for data-annotations 6.0.1
File Interpreter ABI Platform Actions
data_annotations-6.0.1-py3-none-any.whl Python 3 none any Details for data_annotations-6.0.1-py3-none-any.whl

Total release size: 243.8 kB

Release files / data_annotations-6.0.1.tar.gz

Download URL data_annotations-6.0.1.tar.gz
Size 105.7 kB
Tags Source
SHA-256 checksum
How to use checksums
56660d1b967bc14296edb9a77e4c0862aef85ebcc10d8b29b19b71a1cde78c6d
BLAKE2b-256 checksum
How to use checksums
173de2297159d253a01b354a19ca7caf59fdcf723108053ab901d751dcdd4439
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.14.10

Release files / data_annotations-6.0.1-py3-none-any.whl

Download URL data_annotations-6.0.1-py3-none-any.whl
Size 138.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d372d60a9cccc5062308f26ac0f17f5719fad359af1e2705e57e40d12282c119
BLAKE2b-256 checksum
How to use checksums
58648e1d715f9f2614ae1c703ce51fb5abc957864f15fb8896ed791d93d99a56
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.14.10

Security for data-annotations 6.0.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page