data-annotations
data-annotations is a Python package for attaching provenance and structured
descriptions to the files and directories your workflows produce.
It writes plain JSON annotation sidecars that are easy to inspect, archive, and publish with research outputs:
- files use
artifact.ext.annotation.json - directories use
data-annotations.jsonat their root
Optional Markdown README sidecars can be generated for human-readable summaries.
Automatic discovery uses only these current filenames. Explicit Python paths and the CLI
--manifest option can load canonical schema 9 content under any filename; legacy
filenames do not make legacy content compatible.
Documentation
The full documentation is organized as a Diátaxis site.
Other links:
Version 5.0.0 is the first stable public release. Releases through 4.1.1 were unsupported beta snapshots; see versioning and compatibility for the SemVer guarantees and the permanent schema 9 archival baseline.
The next release writes
annotation_version: "10", which adds execution-aware provenance. Tools older than 6.0.0 reject those documents; see versioning and compatibility for the migration and the--annotation-version 9escape hatch.
Installation
Install the core library from PyPI:
pip install data-annotations
Or add it to a project with uv:
uv add data-annotations
Install CLI support when you want the data-annotations command:
pip install "data-annotations[cli]"
uv add "data-annotations[cli]"
Quick start
Decorate a function that writes an artifact. When the function runs, data-annotations
records provenance and writes the JSON sidecar.
from pathlib import Path
from data_annotations.annotations import record_file_annotation
from data_annotations.description import FieldDefinition
@record_file_annotation(
title="Participant Cohort",
summary="Participant-level cohort assignments.",
fields=[
FieldDefinition(
name="participant_id",
data_type="string",
summary="Stable participant identifier.",
required=True,
nullable=False,
),
],
primary_key=["participant_id"],
artifact_kind="dataset",
write_readme=True,
)
def write_participants(artifact_path: Path, input_path: Path) -> Path:
participant_ids = [
line.strip()
for line in input_path.read_text(encoding="utf-8").splitlines()[1:]
if line.strip()
]
artifact_path.parent.mkdir(parents=True, exist_ok=True)
artifact_path.write_text(
"participant_id\n" + "\n".join(participant_ids) + "\n",
encoding="utf-8",
)
return artifact_path
artifact_path = Path("outputs") / "participants.csv"
write_participants(
artifact_path=artifact_path,
input_path=Path("data/raw/participants.csv"),
)
This writes:
outputs/participants.csv
outputs/participants.csv.annotation.json
outputs/participants.csv.README.md
CLI
The CLI supports running annotated commands, retrospective annotation, provenance inspection, source recovery, and sanitized publish bundles.
data-annotations run --answers summary.yaml -- Rscript summarize.R raw.csv summary.csv
data-annotations annotate file path/to/participants.csv --write-readme
data-annotations annotate directory path/to/run-001 --recursive
data-annotations annotate tui
data-annotations provenance match path/to/participants.csv
data-annotations provenance chain path/to/participants.csv
data-annotations provenance checkout path/to/participants.csv
data-annotations publish path/to/run-001 path/to/publish-bundle
Development
From a source checkout (assuming you have Task installed):
task install
task lint
task type-check
task test
Build or preview the documentation site:
task docs-build
task docs-serve
TUI tests
TUI tests use the shared readiness helpers in tests/tui/helpers.py to wait for
the selected form's target, focus, and layout before asserting UI state. The waits
process messages and refreshes, polling every 50 ms with a five-second deadline
that reports the expected condition and current focus. Use these helpers instead
of assuming that one pilot.pause() completes a selection or layout update.
Python runtime upgrades
The local .python-version pin selects Python 3.14. CI sets UV_PYTHON from each
job's PYTHON_VERSION so uv uses the requested matrix version instead of the local
pin. After synchronization, each job prints its interpreter version and executable
and checks that the version matches the job configuration.
Renovate groups updates to .python-version and the CI PYTHON_VERSION variable.
Upgrades to a new Python release line, such as 3.14 to 3.15, require approval in
Renovate's Dependency Dashboard before it creates a branch or merge request. These
upgrades use a separate python-runtime upgrades group; patch updates within a
release line retain the python-runtime group and do not require dashboard approval.
Before approving a new release line, check dependency compatibility and wheel
availability for the CI image. When adopting it, add the new Python version to the
CI test matrix and update the supported Python classifiers in pyproject.toml.
Metadata
Release files for data-annotations 6.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| data_annotations-6.0.1.tar.gz | 105.7 kB | Without provenance | Details for data_annotations-6.0.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| data_annotations-6.0.1-py3-none-any.whl | Python 3 | none | any | Details for data_annotations-6.0.1-py3-none-any.whl |
Total release size: 243.8 kB
Release files / data_annotations-6.0.1.tar.gz
| Download URL | data_annotations-6.0.1.tar.gz |
|---|---|
| Size | 105.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
56660d1b967bc14296edb9a77e4c0862aef85ebcc10d8b29b19b71a1cde78c6d
|
|
BLAKE2b-256 checksum How to use checksums |
173de2297159d253a01b354a19ca7caf59fdcf723108053ab901d751dcdd4439
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.14.10
|
Release files / data_annotations-6.0.1-py3-none-any.whl
| Download URL | data_annotations-6.0.1-py3-none-any.whl |
|---|---|
| Size | 138.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
d372d60a9cccc5062308f26ac0f17f5719fad359af1e2705e57e40d12282c119
|
|
BLAKE2b-256 checksum How to use checksums |
58648e1d715f9f2614ae1c703ce51fb5abc957864f15fb8896ed791d93d99a56
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.14.10
|
Security for data-annotations 6.0.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.