molcrafts-ci

Shared GitHub-native engineering infrastructure for MolCrafts: collect, validate, track, publish, and present developer-facing CI data.

GitHub: MolCrafts/molcrafts-ci

Domain peers:

• molcrafts-bench
• molcrafts-molrec

These are standalone developer-facing repositories (MolCrafts is not a monorepo).

What it is

A snapshot is infrastructure metadata (Manifest) plus a record-specific payload. This repository owns the tests and coverage payload shapes; a benchmark or domain record is built by its own producer and only submitted here.

Install (Python)

pip install molcrafts-ci
# or editable:
pip install -e ".[dev]"

Downstream code should use the short import:

from pathlib import Path
import molci as mci

snap = mci.Snapshot(
    manifest=mci.Manifest(
        record="benchmark",
        source=mci.Source(repository="MolCrafts/molpy", commit="abc1234"),
        producer="pytest-benchmark",
        tracking=mci.Tracking(enabled=True, generation=1),
    ),
    payload={"metrics": {"mean_ns": 12.5}},
)
mci.ingest_snapshot(Path("data"), "molpy", snap)

(molcrafts_ci remains the canonical package path; molci is the public alias.)

CLI

molci validate-snapshot path/to/snapshot.json
molci ingest path/to/snapshot.json --project molpy --data-root data
molci snapshot --out out/ --junit junit.xml --coverage coverage.json

molci snapshot

Turns a test run's native output into tests and coverage snapshots, so a producer repository does not carry its own adapter:

# Python: pytest --junitxml=junit.xml --cov-report=json:coverage.json
molci snapshot --out out/ --junit junit.xml --coverage coverage.json --track

# Rust: RUSTFLAGS="-C instrument-coverage" cargo test --lib | tee cargo-test.log
#       llvm-profdata merge … && llvm-cov export -format=lcov …
molci snapshot --out out/ \
  --cargo-test cargo-test.log \
  --coverage lcov.info --coverage-format lcov \
  --coverage-producer llvm-cov --track

--junit and --cargo-test are mutually exclusive. Provenance comes from the GITHUB_* environment; --track refuses to run without a real GITHUB_SHA, so a placeholder commit can never enter published history.

These two payload shapes are infrastructure, not domain semantics: the frontend already reads them without being told which record it is looking at. A benchmark payload is not built here — that belongs to its domain.

Submitting data from another repository

A producer repository (molpy, molrs, molcrafts-molrec, …) writes Snapshot JSON during its own CI and hands it to one Action. That Action validates it, ingests it into this repository's data branch, and pushes — and that push is what makes Cloudflare Pages rebuild the site. There is nothing to trigger afterwards.

End-to-end (tests + coverage)

# .github/workflows/ci.yml in the producer repository
permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install molcrafts-ci pytest pytest-cov
      - run: pytest --junitxml=junit.xml --cov --cov-report=json:coverage.json
      - run: molci snapshot --out .ci-out --junit junit.xml --coverage coverage.json --track
      - uses: MolCrafts/molcrafts-ci/actions/submit@master
        with:
          snapshot-path: .ci-out/*.json
          project: molpy
          ssh-key: ${{ secrets.MOLCRAFTS_CI_SSH_KEY }}

A domain producer that already builds its own Snapshot JSON skips molci snapshot and only calls submit:

- uses: MolCrafts/molcrafts-ci/actions/submit@master
  with:
    snapshot-path: out/*.json
    project: molpy
    ssh-key: ${{ secrets.MOLCRAFTS_CI_SSH_KEY }}

snapshot-path takes a path, a glob, or a newline-separated list, so one run can submit its benchmark, coverage and test records together. Pin @master to a release tag once one exists.

This repository is its own first producer

molcrafts-ci publishes its own tests and coverage records through the same Action, in .github/workflows/ci.yml:

- uses: ./actions/submit
  with:
    snapshot-path: .ci-out/snapshots/*.json
    project: molcrafts-ci
    token: ${{ github.token }}   # same repository — see below

So the publish path runs on every push to master, not only when a downstream repository adopts it.

Two things make the self-hosted case different from a downstream one:

• GITHUB_TOKEN is enough, because the push goes to the repository the workflow already runs in. No App and no PAT are involved.
• There is no ingest loop. GitHub does not start a workflow from a push made with GITHUB_TOKEN, so the ingest commit cannot trigger another CI run that would ingest again. Cloudflare Pages builds from its own webhook, which is not subject to that rule, so the site still redeploys.

A pull request builds the same two snapshots with tracking.enabled false, so they are validated and uploaded as an artifact but never published. Only the Python suite is published; the site job's tests are not.

The measuring steps run even when a gate above them failed — a run whose tests went red is exactly the run whose tests record is worth keeping.

What decides whether anything is published

manifest.tracking.enabled, in the snapshot itself — not an Action input:

So a pull-request run and a merge run can use the identical step; only the snapshot the producer wrote differs.

Where the data goes

Snapshots are ingested into the data branch, not master. That branch is orphaned, its root is the data root (index/, snapshots/, index-listing.json), and nothing but this Action writes to it. Keeping it off master is what keeps ingest commits out of the code history and keeps a clone of the source from carrying every snapshot ever published.

index-listing.json lists every index/<project>/<record>.jsonl path and each project's newest timestamp (published), so the site can open on whatever ran last without fetching every index over HTTP.

The site reads that branch at runtime from raw.githubusercontent.com, so a published snapshot appears within the CDN's five-minute cache with no rebuild, and the deployment does not grow with history.

Credentials

Pushing into this repository is a cross-repository write, and a workflow's GITHUB_TOKEN is scoped to the repository it runs in — it cannot do this. Use one of:

• a deploy key (what the MolCrafts producers use): pass the private half as ssh-key. It is the narrowest credential that works — it reaches exactly one repository and can only push git, unlike a PAT that carries the whole repo scope.
• a MolCrafts GitHub App: pass app-id and private-key; the Action mints a token that expires with the run. The best posture, but a GitHub App cannot be created from the API, so it takes a trip to the browser.
• a fine-grained PAT with contents: write on molcrafts-ci, as token:.

The Action fails with an explicit message when a tracked snapshot arrives without a credential, rather than going green having published nothing.

The Actions it calls run on Node 24, so a self-hosted runner must be at least Actions Runner 2.327.1; GitHub-hosted runners already are. Behind an HTTP proxy, create-github-app-token v3 needs NODE_USE_ENV_PROXY=1 on the step.

If the index branch is protected, the App or PAT identity needs to be allowed to bypass the rule — otherwise every ingest is rejected five times and the producer's job fails.

Re-runs, retries and two producers at once

The ingest is idempotent. A snapshot id is derived from its source, the stored file is immutable, and an index entry is appended only when that id is not already indexed, so:

• re-running a job that already published is a no-op — it pushes nothing;
• if another repository pushes first, submit re-runs the ingest on the updated branch and pushes again (up to five attempts), so neither side's data is lost.

Other inputs

Outputs: count, tracked, ingested, skipped, pushed, commit. The run also gets a job summary naming every snapshot and the commit it landed in.

Ingesting by hand

No Action required — the CLI is the same code path:

molci ingest path/to/snapshot.json --project molpy --data-root data --if-exists skip
git add data && git commit -m "ingest: molpy" && git push

(For the published site, push to the data branch with that branch's root as the data root — the same layout actions/submit uses.)

Frontend (site/)

Built with molcrafts-ui (shadcn registry copy-in). Dev mocks use rspack-plugin-mock (Rsbuild-native) — not hand-served static files.

# regenerate fixtures via molci, then install + run
python3 -m pip install -e ".[dev]"
npm --prefix site run seed:mock   # or: python3 scripts/seed_mock_data.py
cd site && npm install && npm run dev

• npm run dev — intercepts /data/** with site/mock/*.mock.ts
• npm run seed:mock — import molci as mci ingest → .mock-data/ (gitignored) + refreshes site/mock/fixtures.ts. These snapshots carry invented commit SHAs, so they are kept out of the tracked data/ tree.
• npm run dev:data — skip mock plugin; fetch the data branch and serve it locally
• npm run build — reads the data branch at runtime; bundles no data at all

How the browser is organised

• Navigator — projects, ordered by newest publish first (from index-listing.json's published map).
• Overview — situation above (facts, verdict distribution, record table), history below (one reading + delta + sparkline per record).
• Record tab — one generation at a time: header carries commit, CI run, profile and producer; body is PayloadView (tests and coverage both use a MeasureBand).
• Dock — publish log and problems; a row opens that record tab at that generation via the URL.

There is no side inspector. A shared link is ?project=&tab=&snapshot=&profile=.

Register an extra tab without editing the shell:

import { registerPlugin } from "@/plugins/registry";

registerPlugin({
  id: "api",
  label: "API",
  order: 60,
  available: (ctx) => ctx.records.includes("api"),
  Component: ({ project }) => <div>{project.id}</div>,
});

Import the module from src/index.tsx (side-effect registration).

UI sources are synced from the sibling molcrafts-ui checkout via npm run sync-ui.

Publish package

pip install build twine
python -m build
# twine upload dist/*   # when ready for PyPI

Versioning: while the project is 0.x, the minor is the breaking-change axis. manifest.schema_version is taken from that minor, so the snapshot shape and the release move together. See CHANGELOG.md.

Specification

See docs/specification.md.