github-repo-atlas 0.2.0

Automated, read-only auditing and knowledge-base generation for scientific open-source GitHub repositories (structure, docs, FAIR readiness, transparency, attribution, agent-readiness).

github-repo-atlas

A neutral Python CLI, reusable GitHub Action, and agent-skill toolkit for initializing and auditing trustworthy scientific open-source repositories.

github-repo-atlas can scaffold a new repository or check an existing one. It also scans the repositories you can reach on GitHub, extracts their structure and documentation, audits them for scientific transparency, FAIR readiness, attribution quality, reproducibility, and agent-readiness, and generates a rich Markdown knowledge base — one card per repo, plus cross-repo synthesis, a scoreboard, and reusable "vibe coding" lessons.

It is designed for a working scientist who wants (1) a durable technical memory of all their repositories and (2) a library of reusable scientific-transparency templates.

---

What this project does

For every repository it can see, the atlas:

1. Inventories it via the GitHub CLI (gh) → data/raw/repos.json.
2. Clones/updates a local read-only copy in cache/repos/OWNER__REPO/.
3. Extracts structure, docs, dependencies, and git history → data/processed/repos/OWNER__REPO.json.
4. Audits a "Repo Trust Pack" (README, LICENSE, CITATION.cff, CREDITS, FAIR.md, AI_DECLARATION.md, AGENT.md, …) with deterministic heuristics.
5. Scores it across 7 explainable dimensions.
6. Tracks trends across runs — each audit snapshots the scores, so you can see deltas (▲/▼) and trajectories over time.
7. Generates a Markdown card, proposed transparency drafts, a scoreboard, cross-repo synthesis/lessons, a trends page, and a workshop kit for teaching good practices from your own repositories.

Why it exists

Scientific software is only as trustworthy as its documentation, provenance, and attribution. The atlas makes those qualities measurable and visible across a whole portfolio, and lowers the cost of fixing gaps by generating draft transparency files you can review and adopt.

Safety model

This is the most important section. By design, the atlas:

• Never modifies your repositories. It only reads, clones, and fast-forwards (git pull --ff-only) local copies. It never pushes, commits to, or opens PRs against a target repo.
• Never prints or copies secrets. Secret detection is by filename only (e.g. .env, *.pem); file contents are never read or shown.
• Writes everything inside this atlas repo (data/, cache/, docs/, packs/). Target repositories are untouched.
• Skips archived repos and forks by default (opt in with --include-archived / --include-forks).
• Requires no paid APIs and no API keys. The deterministic V1 uses only the GitHub CLI you already authenticate. An LLM enhancement hook exists but is optional and disabled by default (ATLAS_ENABLE_LLM=false).
• Never auto-validates CRediT roles or AI declarations, and never treats a generated draft as authoritative.

Any future ability to open PRs to target repos would be a separate, explicitly opt-in feature — it does not exist in this version.

Requirements

• Python 3.11+
• GitHub CLI (gh), authenticated — the only way the atlas reaches GitHub.
• Optional: Repomix for compact LLM packs (the audit works fine without it).

Python dependencies (installed automatically): typer, pydantic, jinja2, rich, and pytest for tests.

Installation

Install the isolated CLI from PyPI:

pipx install github-repo-atlas
atlas --help

To test the latest unreleased main branch instead:

pipx install "git+https://github.com/dhuzard/github-repo-atlas.git@main"

To contribute from a source checkout:

git clone https://github.com/dhuzard/github-repo-atlas.git
cd github-repo-atlas
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\Activate.ps1
pip install -e ".[dev]"

Quick start: initialize a repository

Create a Python research repository with tests, CI, citation metadata, agent instructions, and FAIR, TRUST, reproducibility, attribution, and AI-use declarations:

atlas init behavioral-analysis \
  --path ./behavioral-analysis \
  --owner my-lab \
  --author "Jane Researcher" \
  --description "Reproducible analysis of behavioral experiment data" \
  --with-data

cd behavioral-analysis
atlas check . --fail-on never

The first check intentionally identifies factual placeholders requiring human review. After completing them from evidence, enable the strict quality gate:

atlas check . --fail-on warning

For the generated file tree, Windows PowerShell commands, GitHub publication, and declaration-review workflow, see Initiate a new repository with the CLI or Codex in VS Code.

Initialize with Codex in VS Code

Open the parent directory in VS Code and ask Codex:

Use the atlas CLI to initialize a new repository named behavioral-analysis
inside ./behavioral-analysis.

Owner: my-lab
Author: Jane Researcher
Description: Reproducible analysis of behavioral experiment data
Include the data declarations.

Initialize git, create a virtual environment, install the generated project,
run pytest, and run `atlas check . --fail-on never`. Report every declaration
that still requires factual human input. Do not invent authorship, funding,
identifiers, validation results, consent, compliance, AI use, or reuse rights.

Codex performs the workflow; the deterministic CLI remains the quality gate.

GitHub authentication

The atlas uses gh, not a token file:

gh auth login          # one-time, interactive
gh auth status         # verify you are logged in

You can check your environment at any time without scanning anything:

python -m atlas.cli doctor

Under the hood the inventory step runs the equivalent of:

gh api '/user/repos?visibility=all&affiliation=owner,collaborator,organization_member&per_page=100&sort=updated' --paginate

If gh is missing or unauthenticated, the atlas fails clearly and tells you to run gh auth login — it never continues with fabricated data.

First run

Start small (5 repos, no Repomix):

python -m atlas.cli full --max-repos 5 --skip-repomix

Then the full run:

python -m atlas.cli full

Scope which accounts/orgs are included via .env (ATLAS_GITHUB_LOGINS) or the --logins flag, e.g. --logins dhuzard,neuronautix.

Individual stages

Every stage is idempotent and runnable on its own:

python -m atlas.cli inventory            # -> data/raw/repos.json
python -m atlas.cli clone                # -> cache/repos/OWNER__REPO
python -m atlas.cli extract              # -> data/processed/repos/*.json (+ audit + scores + trend)
python -m atlas.cli audit                # print Trust Pack summary
python -m atlas.cli generate-cards       # -> docs/repo_cards/*.md
python -m atlas.cli generate-trust-pack  # -> docs/transparency_pack/OWNER__REPO/*.md
python -m atlas.cli synthesize           # -> scoreboard + synthesis + lessons + trends
python -m atlas.cli trends               # print the score trend across runs
python -m atlas.cli workshop             # -> docs/workshop/ teaching kit
python -m atlas.cli backfill-history     # seed trend baselines from existing reports (no re-scan)

Flags: --max-repos N, --include-archived, --include-forks, --skip-repomix, --logins a,b.

Output structure

data/raw/repos.json                     # raw GitHub inventory
data/processed/repos/OWNER__REPO.json    # deterministic per-repo report
cache/repos/OWNER__REPO/                 # local read-only clone
packs/repomix/OWNER__REPO.repomix.md     # optional LLM pack
docs/repo_cards/OWNER__REPO.md           # human-readable repo card
docs/transparency_pack/OWNER__REPO/*.md  # proposed AI-assisted drafts (review!)
docs/audits/scoreboard.md                # portfolio table (with Δ + trend columns)
docs/audits/transparency_audit.md        # ecosystem transparency coverage
docs/audits/trends.md                    # cross-run score trends
docs/synthesis/cross_repo_synthesis.md   # portfolio-level synthesis
docs/lessons/*.md                        # vibe coding lessons + templates
docs/workshop/*.md                       # teaching kit (curriculum, case studies, ...)
data/reports/history/                    # append-only trend snapshots (committable)

How to interpret the scores

Seven deterministic scores, each 0–100 and decomposed into weighted components (visible in every repo card and in data/processed/):

Score	What it measures
Repo Usability	Can a newcomer install, run, and test it from the README?
Agent Readiness	Can an AI coding agent work safely (AGENT.md, tests, CI, clear commands, no secret risk)?
Transparency	Coverage of the Trust Pack documentation set.
FAIR Readiness	Findable / Accessible / Interoperable / Reusable signals.
Attribution Quality	Contributors, CRediT roles, software/funding credit, AI disclosure.
Reproducibility	Pinned environment, tests, CI, documented run steps.
Scientific Trust	Validation, limitations, reproducibility, citation, AI disclosure.

Important: these scores measure documentation and engineering hygiene — a proxy for trustworthiness. They are not a certification of scientific validity. A high score means "the repo plausibly documents the right things", never "the science is correct".

Cross-run trend tracking

Every run appends a compact snapshot of each repo's scores to data/reports/history/ (per-repo *.jsonl + a portfolio runs.jsonl). From that history the atlas computes:

• Δ since last audit on the scoreboard and each card (▲/▼/—).
• Sparklines (▁▂▃▄▅▆▇█, scaled 0–100) showing each repo's trajectory.
• A trends page (docs/audits/trends.md) with the portfolio average over time, a per-repo trajectory table, and "most improved" / "declining" lists.

History is the one artefact that cannot be regenerated from a fresh scan, so it is git-committable by default (the rest of data/ is ignored). Identical consecutive snapshots are de-duplicated, so re-running without changes adds no noise. Use --no-history to skip recording, or trends to print the table.

Seeding from existing reports: if you have reports from before trend tracking existed, run python -m atlas.cli backfill-history once to establish baselines without re-scanning — the next audit will then show real deltas.

Teaching & workshops (workshop command)

python -m atlas.cli workshop turns your audited portfolio into a ready-to-teach kit in docs/workshop/, so you can run trainings on good "vibe coding" practices using your own repositories as the evidence:

File	What it is
README.md	Index + suggested formats (90-min, half-day, team retro).
curriculum.md	6 modules (runnable → agent-ready → reproducible → FAIR → attribution → AI/ethical debt), each with a real exemplar repo, a contrast repo, a reusable template, and an exercise.
case_studies.md	Your flagship repo, your biggest opportunity, and improvement stories drawn from trends.
playbook.md	The 7 durable principles + quick wins ranked by how often they apply across your repos.
exercises.md	Time-boxed hands-on tasks that produce a measurable score change.
slides.md	A Marp/Reveal-compatible slide outline.

Why teach from your own repos: participants can open the repo, see the score, see exactly which file is missing, add it, and watch the score move on the next run. The maturity ladder (Level 1 Ad hoc → Level 5 Exemplary) and the trends page make that feedback loop the centrepiece of the session.

The teaching loop: measure → pick the lowest dimension → apply that module's template → re-audit → show the delta. Run one improvement live and the trends page records your first "improvement story" during the session.

Improving existing repos & starting new projects

The atlas is read-only by default, but it ships two explicit, opt-in commands to act on what it finds. Neither ever commits or pushes — you review and commit.

Check any local repository — check

Run the declaration quality checker without inventorying or cloning GitHub:

# Human-readable report; only unresolved placeholders fail by default
atlas check /path/to/repo

# CI annotations + JSON; missing/weak recommended declarations fail
atlas check /path/to/repo \
  --format github \
  --output declaration-quality.json \
  --fail-on warning

The checker recognizes root fair.md / trust.md, uppercase variants, and the existing docs/FAIR.md / docs/TRUST.md Trust Pack layout. It checks declaration structure and unresolved placeholders; it does not certify scientific correctness, FAIRness, security, or epistemic trust.

Start a new project the right way — init

Scaffold a new repository that is agent-ready, FAIR and transparent from the first commit (it would score highly on the atlas immediately):

atlas init my-new-tool --owner dhuzard --description "What it does"

This creates README, LICENSE, CITATION.cff, CREDITS.md, CONTRIBUTING.md, SECURITY.md, the full docs/ Trust Pack (FAIR, AI_DECLARATION, DESIGN, TRUST, REPRODUCIBILITY, LIMITATIONS, ETHICAL_DEBT, AGENT), a root AGENTS.md, a Python package + smoke test, a CI workflow, .gitignore and .env.example. Add --with-data for DATA/MODEL cards + governance. It also includes a declaration quality workflow using this repository's reusable action. The scaffold's own smoke test passes out of the box; the declaration workflow intentionally stays red until you replace the factual TODOs. Until a v1 tag is published, the generated workflow uses @main and includes a reminder to pin a release or SHA.

Improve an existing repo — apply-pack

After generate-trust-pack, materialize a repo's AI-assisted drafts into your working copy of that repo (not the atlas's clone cache) for review:

# 1. preview (default: writes nothing)
python -m atlas.cli apply-pack --repo dhuzard/HCMO --into /path/to/your/HCMO
# 2. actually add the missing files
python -m atlas.cli apply-pack --repo dhuzard/HCMO --into /path/to/your/HCMO --write
# then, in YOUR repo:
git checkout -b add-trust-pack && git add -A && git commit -m "Add Trust Pack drafts"
# open a PR yourself — the atlas never pushes

Safety: dry-run unless --write; only adds missing files unless --force; refuses to write into cache/repos/; never runs git. The drafts keep their "AI-assisted — requires human review" banner so you review before adopting.

How to use the generated transparency files

docs/transparency_pack/OWNER__REPO/ contains proposed drafts for missing or weak Trust Pack files (FAIR.md, AI_DECLARATION.md, DESIGN.md, TRUST.md, REPRODUCIBILITY.md, CREDITS.md, ETHICAL_DEBT.md, AGENT.md). Each begins with:

Status: AI-assisted draft generated from repository structure and metadata. Requires human review before being treated as authoritative.

Workflow: review → edit → copy into the target repo yourself. The atlas never places them there for you. Blank, reusable versions of all templates live in templates/trust_pack/.

Reusable quality gate

Other repositories can use the composite GitHub Action after checking out their source:

permissions:
  contents: read

steps:
  - uses: actions/checkout@v5
  - uses: dhuzard/github-repo-atlas@v0.2.0
    with:
      fail-on: warning

Update the version deliberately when adopting a new release. Security-sensitive consumers should pin the action to a reviewed full commit SHA. The CLI remains the portable option for GitLab, local pre-commit, HPC, and non-GitHub environments.

Framework source refresh

framework/sources.toml is the source-of-truth registry for every recommended Markdown declaration. FAIR and TRUST are upstream-governed by Neuronautix/FAIR.md and Neuronautix/TRUST.md; the other declarations currently point to their locally governed templates. Each source can later move to an independent specification repository without changing the pipeline.

# Exit 0: current; 2: check failed; 3: reviewable drift
atlas framework-check \
  --report framework-source-report.md \
  --json-output framework-source-report.json \
  --snapshot-dir .framework-sources

# Only after implementation + human review
atlas framework-check --accept

The weekly and manually triggered framework-refresh.yml workflow builds a reproducible evidence bundle. On explicit manual request, agentic_proposal=true sends that bundle to the official Codex GitHub Action in read-only mode and opens its proposal as an issue. It never auto-accepts a new specification, edits standards, or merges a change. Configure the OPENAI_API_KEY repository secret only if you want that optional step.

Codex and Claude skills

Two portable Agent Skills are checked into both discovery locations:

• audit-repo-declarations — assess, initialize, or improve a repository without inventing scientific claims.
• refresh-declaration-framework — turn upstream drift into a tested, human-reviewed framework update.

Codex discovers them under .agents/skills/; Claude Code discovers the byte-identical copies under .claude/skills/. The skills deliberately depend on the CLI's deterministic evidence rather than reimplementing checks in prompts.

For broader distribution, keep the skills in the open SKILL.md format, tag releases of this repository, and package them as a Codex/ChatGPT plugin or Claude plugin only as an additional install channel. The CLI + JSON report + GitHub Action should remain the stable public contract for developers and scientists who do not use either agent.

Community adoption model

The most sustainable split is:

1. Conventions: version FAIR.md and TRUST.md independently with specification, schema, template, changelog, examples, and conformance fixtures.
2. Conformance tooling: publish this atlas as a Python package/CLI and a versioned GitHub Action; keep checks explainable and offline where possible.
3. Agent workflows: distribute the same open Agent Skills for Codex and Claude; agents propose changes, deterministic checks verify them.
4. Governance: use normal issues, RFC pull requests, releases, CITATION.cff, and archived release DOIs. Treat adoption evidence and false-positive reports as inputs to the next specification version.

This avoids making an LLM vendor a prerequisite for using the declarations and gives scientific users a citeable, versioned, reviewable standard.

What is deterministic vs AI-assisted

• Deterministic (this version): the entire pipeline — inventory, extraction, Trust Pack audit, all 7 scores, cards, scoreboard, synthesis, and the structure of the transparency drafts. Given the same repos, you get the same output. No LLM is involved.
• AI-assisted (optional, off by default): the framework refresh workflow can ask Codex to review a deterministic drift bundle and open a proposal issue. It requires an explicit manual trigger and OPENAI_API_KEY; it cannot accept or merge a standards update.

Limitations

• The audit uses filename + keyword heuristics. It detects the shape of good documentation, not its correctness.
• Secret detection is filename-only — it will miss secrets hidden inside otherwise-innocuous files, and may flag harmless files by name.
• Contributor roles are inferred from git authorship and must be human-validated; they are never asserted as fact.
• Scores are proxies; treat them as a to-do prioritiser, not a grade.
• gh rate limits and repository size affect scan time.

Roadmap

• ✅ Trend tracking across runs (score deltas over time) — done.
• ✅ Workshop / teaching kit derived from your portfolio — done.
• ✅ init scaffolder for new projects + apply-pack for existing repos — done.
• ✅ Portable check command + reusable GitHub Action — done.
• ✅ Upstream registry, reviewed lock, scheduled drift evidence, and opt-in Codex proposal — done.
• ✅ Cross-compatible Codex/Claude Agent Skills — done.
• Optional, opt-in LLM enrichment of draft prose (still human-reviewed).
• Richer FAIR signals (DOI/Zenodo detection, metadata standard recognition).
• Trend charts (SVG sparklines) embedded in cards.
• Optional --pr flag on apply-pack to open a review PR via gh (opt-in).
• Per-language deepening of install/run/test inference.

Development

python -m pytest                 # run the test suite
python -m atlas.cli doctor       # check gh / repomix availability

License

MIT (see LICENSE). This atlas reports on other repositories; it makes no claim about their licenses beyond what it reads from their metadata.