research-repo-tools

Shared setup, checks, notebooks, performance evidence and release tooling for research software repositories. Install one pinned Python package and invoke it through your repository's Just recipes. Keep scientific algorithms, benchmark case selections, native builds and deployment policy in your own project.

Start with installation, try the first workflow, then choose a capability. Package development belongs in CONTRIBUTING.md.

Release availability

This README describes current source, including work intended for v0.2.5. The shared-environment Semgrep and representation/retention policy need v0.2.5. Availability-aware benchmark suites, acquisition progress and checkpoint/resume also require a supplying v0.2.5 release; see the suite contract. The quickstart uses the published 0.2.3 package. Newer examples are marked where a supplying release is required: prepared performance stages, parallel Criterion suites, shared release-evidence handoffs, command timing, optional CI reuse, and the notebook/stream refinements need v0.2.4.

An upstream merge, local wheel or passing source test does not publish a package. Before adopting a new capability, verify its exact PyPI release, update your consumer's pin and lockfile, and run its integration checks. Keep working adapters until that adoption succeeds. Package publication and consumer adoption are separate steps. Migration guidance covers compatibility changes; the generated CHANGELOG.md records release history.

Install with uv

Use an existing consumer Git repository; you do not need to clone this package. Install uv and Git first. Recipes require a POSIX sh on PATH, including on Windows: add Git for Windows' bin directory, typically C:\Program Files\Git\bin. uv supplies managed Python; the published package requires Python 3.15 or newer.

From the consumer root, add the exact published package:

uv add --group tooling --no-sync "research-repo-tools==0.2.3"

Include tooling from dev, retaining your other dependencies. Set .python-version to a compatible selection such as 3.15. Keep any stricter application Python constraints; use the adoption workflow if the project currently requires an older interpreter.

[dependency-groups]
tooling = ["research-repo-tools==0.2.3"]
dev = [{include-group = "tooling"}]

Declare the uv version you installed as tool.uv.required-version = "==X.Y.Z". The toolchain guide explains Rust and optional tool declarations; a Python-only consumer can omit them. Refresh the lock, then export the matching installed Just template:

uv lock
uv run --locked --only-group tooling research-repo-tools templates justfile --output justfile

For an existing justfile, omit --output justfile and merge the printed recipes; the exporter refuses to overwrite files. Review and retain the manifest, lockfile, Python selector and recipes together.

Run the one-time setup:

uv run --locked --managed-python --only-group tooling research-repo-tools setup

Setup verifies the declarations, installs a pinned user-level Just and declared tools, configures PATH and synchronizes the locked development environment. Open a new terminal if PATH changed. A failed step stops setup with a diagnostic; fix that prerequisite and rerun. Installation and recovery explains ownership, native build prerequisites and platform behavior.

Semgrep consumers of v0.2.5 must request the semgrep extra on their shared package pin, retain the shared Python 3.15 selector, and adopt the package's shared dependency overrides before locking. Scans and fixture checks use that same environment. A package upgrade does not rewrite copied Just recipes; merge the relevant exported recipes as part of each consumer adoption and verify its integration before removing temporary adapters.

First workflow

After setup, list your recipes and check Markdown source lines in the consumer:

just help
just files check-lines --limit 160 --include '*.md' --exclude CHANGELOG.md

This read-only check needs no optional scanner or notebook dependencies. It selects tracked and nonignored Markdown, counts every raw line (including tables and code), and returns zero for a clean inventory. Long lines produce file:line: line length N exceeds 160 and status 1. Fix the selected source or declare your intended selection and limit, then rerun.

Use just ... for routine work; no environment activation is needed. Recipes select the locked package internally. The underlying CLI and Python APIs remain available for integrations.

Common workflows

These are independent commands; run only the workflow you intend. Some require consumer configuration or optional dependencies described in their guides.

The complete command and capability reference is organized lexicographically. just help remains the authority for the recipes actually merged into your project. The packaged templates provide thin wrappers; consumers extend checks with their own Rust, scientific and deployment requirements.

Capability index

Choose a task guide for execution order and outcomes; use its linked reference for exhaustive arguments, configuration and Python contracts.

Git inspection

The index-preserving inspection API targets v0.2.5. After that release is published, pin it before replacing local inspection wrappers:

from pathlib import Path

from research_repo_tools.git_inspect import git_diff_paths_bytes, git_status_bytes

root = Path.cwd()
status = git_status_bytes(root).stdout
staged = git_diff_paths_bytes(root, cached=True).stdout.split(b"\0")[:-1]
worktree = git_diff_paths_bytes(root, base="HEAD", paths=["src"]).stdout.split(b"\0")[:-1]

Queries preserve the selected index bytes and return NUL-delimited byte output with literal path selection. See the inspection contract for configuration, deadlines, error handling and guarantee limits.

Notebooks

Add the notebooks extra at your exact tooling version to a separate notebook dependency group. Declare analysis packages and the required Ruff/ty checkers, refresh the lock, then follow notebook setup:

just notebook-sync
just notebook-lint notebooks/analysis.ipynb
just notebook-execute notebooks/analysis.ipynb

Execution preserves source notebooks and writes executed copies plus JSON reports under target/notebooks. A failed cell returns nonzero and identifies the cell. Consumers retain scientific assertions and fast/slow selection. Interactive JupyterLab, explicit source reset and source-policy checks are in the same guide.

Performance

For v0.2.4, configure the installed performance-release.toml template with your measurement, retained-report and document-publication policies. Declare and ignore its scratch directory; keep the report archive outside scratch. Then run:

just performance-prepare tooling/performance-release.toml
just performance-release tooling/performance-release.toml
just performance-readme tooling/performance-release.toml --preview
just performance-readme tooling/performance-release.toml

For schema-5 measurement declarations, use a new ignored checkpoint directory whose parent already exists. Resume explicitly after an interruption:

just performance-release tooling/performance-release.toml --checkpoint target/performance-checkpoint
just performance-release tooling/performance-release.toml --checkpoint target/performance-checkpoint --resume

Completed paired suites retain their original raw bytes and sessions. An incomplete pair is recollected in full. --progress json emits versioned status events on stderr; --progress none disables status. Report and README retries continue to use retained evidence without measuring again.

Preparation saves independently selected release tags and full commits. Measurement runs the gates and benchmarks, then retains/promotes the complete evidence. README publication only consumes retained evidence: repair a presentation failure and retry that last stage without measuring again. The performance guide explains source exports versus worktrees, common versus independent harnesses, parallel suite assembly, ratio eligibility and first-use report state.

The v0.2.5 benchmark reuse API derives release archives from one accepted complete-run v4 acquisition. Supply a trusted, independently prepared reuse policy, then plan and publish the archive without invoking benchmarks:

from pathlib import Path

from research_repo_tools.benchmark_reuse import plan_reuse_archive
from research_repo_tools.publication import publish_publication

candidate = plan_reuse_archive(
    Path.cwd(), "target/run.json", "target/evidence.json",
    "tooling/reuse-policy.json", "target/benchmark-release.tar.gz",
)
publish_publication(candidate)

This requires exact tagged source measurements. If README updates must precede the final release commit, retain the existing two-acquisition workflow. The derived archive preserves original evidence and does not authenticate CI origin; attachment and approval retain the existing release safeguards. Verify the supplying PyPI release before adoption.

Releases

Follow the ordered release procedure and matching installed RELEASING.md template. Exact-commit checks are the default. CI reuse is an explicit reviewed option with its own eligibility and revalidation; package and artifact checks still apply. Approval, registry publication, attachment uploads and downstream adoption remain distinct actions.

Contributing

See CONTRIBUTING.md for package development, tests and maintainer commands. Supported interfaces defines the consumer compatibility boundary.

Citation

If you use research-repo-tools in your work, cite the software using CITATION.cff. Include the version used so others can identify the same release. The Zenodo concept DOI covers all versions; use the version-specific DOI from the corresponding Zenodo record when citing an exact release.

License

BSD-3-Clause. See LICENSE.