Skip to main content

data-annotations

PyPI Documentation License CI

data-annotations is a Python package for attaching provenance and structured descriptions to the files and directories your workflows produce.

It writes plain JSON annotation sidecars that are easy to inspect, archive, and publish with research outputs:

  • files use artifact.ext.annotation.json
  • directories use data-annotations.json at their root

Optional Markdown README sidecars can be generated for human-readable summaries.

Automatic discovery uses only these current filenames. Explicit Python paths and the CLI --manifest option can load canonical schema 9 content under any filename; legacy filenames do not make legacy content compatible.

Documentation

The full documentation is organized as a Diátaxis site.

Other links:

Version 5.0.0 is the first stable public release. Releases through 4.1.1 were unsupported beta snapshots; see versioning and compatibility for the SemVer guarantees and the permanent schema 9 archival baseline.

The next release writes annotation_version: "10", which adds execution-aware provenance. Tools older than 6.0.0 reject those documents; see versioning and compatibility for the migration and the --annotation-version 9 escape hatch.

Installation

Install the core library from PyPI:

pip install data-annotations

Or add it to a project with uv:

uv add data-annotations

Install CLI support when you want the data-annotations command:

pip install "data-annotations[cli]"
uv add "data-annotations[cli]"

Quick start

Decorate a function that writes an artifact. When the function runs, data-annotations records provenance and writes the JSON sidecar.

from pathlib import Path

from data_annotations.annotations import record_file_annotation
from data_annotations.description import FieldDefinition


@record_file_annotation(
    title="Participant Cohort",
    summary="Participant-level cohort assignments.",
    fields=[
        FieldDefinition(
            name="participant_id",
            data_type="string",
            summary="Stable participant identifier.",
            required=True,
            nullable=False,
        ),
    ],
    primary_key=["participant_id"],
    artifact_kind="dataset",
    write_readme=True,
)
def write_participants(artifact_path: Path, input_path: Path) -> Path:
    participant_ids = [
        line.strip()
        for line in input_path.read_text(encoding="utf-8").splitlines()[1:]
        if line.strip()
    ]
    artifact_path.parent.mkdir(parents=True, exist_ok=True)
    artifact_path.write_text(
        "participant_id\n" + "\n".join(participant_ids) + "\n",
        encoding="utf-8",
    )
    return artifact_path


artifact_path = Path("outputs") / "participants.csv"
write_participants(
    artifact_path=artifact_path,
    input_path=Path("data/raw/participants.csv"),
)

This writes:

outputs/participants.csv
outputs/participants.csv.annotation.json
outputs/participants.csv.README.md

CLI

The CLI supports running annotated commands, retrospective annotation, provenance inspection, source recovery, and sanitized publish bundles.

data-annotations run --answers summary.yaml -- Rscript summarize.R raw.csv summary.csv
data-annotations annotate file path/to/participants.csv --write-readme
data-annotations annotate directory path/to/run-001 --recursive
data-annotations annotate tui
data-annotations provenance match path/to/participants.csv
data-annotations provenance chain path/to/participants.csv
data-annotations provenance checkout path/to/participants.csv
data-annotations publish path/to/run-001 path/to/publish-bundle

Development

From a source checkout (assuming you have Task installed):

task install
task lint
task type-check
task test

Build or preview the documentation site:

task docs-build
task docs-serve

Metadata

Release files for data-annotations 6.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for data-annotations 6.0.0
File Size Uploaded Provenance Actions
data_annotations-6.0.0.tar.gz 100.4 kB Without provenance Details for data_annotations-6.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for data-annotations 6.0.0
File Interpreter ABI Platform Actions
data_annotations-6.0.0-py3-none-any.whl Python 3 none any Details for data_annotations-6.0.0-py3-none-any.whl

Total release size: 237.8 kB

Release files / data_annotations-6.0.0.tar.gz

Download URL data_annotations-6.0.0.tar.gz
Size 100.4 kB
Tags Source
SHA-256 checksum
How to use checksums
addd4e668b3373c027d8f21deed0e79507b44a64f03ba603583168a1113ff5d7
BLAKE2b-256 checksum
How to use checksums
d811d84731e5444fb4d093a4591d6e7a38ed2f0650ffe4e2a5866c5f68863188
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.14.7

Release files / data_annotations-6.0.0-py3-none-any.whl

Download URL data_annotations-6.0.0-py3-none-any.whl
Size 137.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
91f260252b17c73ef150998944023f3ba865e2f014ebee61d24e2bec2ef32b5c
BLAKE2b-256 checksum
How to use checksums
75c8046fd48bef6a204eef593857be5010178758273d5c8e14610631a8b5bd42
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.14.7

Security for data-annotations 6.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page