research-graph

Contract-gated research verification · public beta

Trace every artifact, verify every handoff, and bound every revision.
A provider-neutral integrity layer for multi-agent research pipelines.

Quickstart   ·   Architecture   ·   Commands   ·   Scope   ·   Live site

What is it?

research-graph is a contract-gated verification layer for multi-agent research pipelines. It reads a directory of versioned artifacts, validates every one against a JSON Schema, walks the provenance hash chain, checks that the reviewer was not the producer, and returns an exit code. next --execute and challenge each launch one configured subscription CLI and then return control; there is no scheduler, continuous agent loop or model API client.

It makes four mechanical properties visible: required artifacts exist, their schemas and hashes agree, recorded reviewer and producer identities are separated, and every revision route has a budget. It does not decide whether the research question, method, interpretation or conclusion is scientifically correct.

Who is it for?

It is for technical researchers and mixed-experience research teams that care about traceable outputs but should not need to learn the artifact JSON format to start. The local UI presents the run as an evidence desk; human and final decisions remain terminal-only, and the CLI remains available for every action, scripts and CI.

Use it as an integrity and provenance layer around a research workflow. Do not use it as evidence that models were orchestrated, reviewers were epistemically independent, or a manuscript is publication-ready.

Two minutes: an empty directory to a governed study, in a browser

One command starts the local application. Everything a first study needs — the demo, the guided setup, provider detection, preflight, the run itself — happens in the browser from there.

mkdir my-research-study
cd my-research-study
git init

uv tool install rgraph==0.4.1
rgraph ui                          # http://127.0.0.1:8765, bound to this computer

With no study in the directory, rgraph ui opens a launcher rather than an error: try the bundled demo, start a new study, reopen a recent one, or open one by path. The new-study wizard asks ten short questions in ordinary language, detects the provider CLIs you already have, previews the exact destination before writing anything, and creates a sealed, governed run. You never type a provider identifier, a run path, or a line of JSON.

The terminal remains the scriptable interface, the bootstrap, and — deliberately — the only place a human decision is recorded:

rgraph demo                       # 30-second synthetic tour; exits 0
rgraph setup                       # choose the six provider/model assignments
rgraph doctor --probe-models       # small real call per distinct model
rgraph init                        # create and seal the study setup
rgraph ui --run path/to/run        # open one specific study

Read each screen's Next action rather than memorising the sequence. doctor checks executables, PATH, login, assignment, capabilities and gate viability. Without --probe-models, model names are deliberately UNVERIFIED; the CLI has no provider-neutral model catalogue it can honestly treat as current.

Human decisions remain interactive and attributable. The CLI refuses piped approval; the browser only displays the current decision state, what was and was not checked, the responsible person, and the exact terminal command — with a copy button. It cannot record a human gate or final decision, and there is no /api/decide or /api/review route to add one to. A TTY does not authenticate the self-declared name, so keep decide and review outside agent allowlists and restrict write access when that distinction matters.

Provider execution is always previewed and separately approved; each approval is single-use and bound to the exact command, prompt, inputs and expected outputs. An approved plan runs as a child process of the local application, not in a terminal you type into: the browser shows its state, elapsed time and a live redacted transcript, and can stop it. One provider execution runs per study at a time. A process that exits 0 has not thereby produced valid artifacts — the console reports process state and artifact validation as separate stages.

Try the verifier in 30 seconds

uv tool install rgraph==0.4.1
rgraph demo --scenario 1

The clean scenario exits 0 after nine gates pass. It checks artifact presence, JSON Schema conformance, the SHA-256 provenance chain, recorded producer/reviewer separation, gate prerequisites and revision budgets. Its final line still states that scientific correctness was not determined.

The bundled example-run is a synthetic fixture, not evidence of a real multi-agent run. Its data, statistics, DOIs and hash chain are real; its provider identities and reviewer decisions are illustrative.

uv fetches a compatible Python and isolates the tool from other environments. Install uv first if needed.

macOS or Linux:

curl -LsSf https://astral.sh/uv/install.sh | sh

Windows PowerShell:

irm https://astral.sh/uv/install.ps1 | iex

To try the clean scenario without installing the tool, run:

uvx --isolated \
  --from rgraph==0.4.1 \
  rgraph demo --scenario 1

From a checkout

For working on the kit itself, use the platform-specific activation command.

macOS or Linux:

python3 --version  # 3.11 or newer
git clone https://github.com/huguryildiz/research-graph
cd research-graph
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .
rgraph demo --scenario 1

Windows PowerShell:

py -3.11 -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install -e .
rgraph demo --scenario 1

If your shell answers command not found: rgraph immediately after that install, it is reading a stale command table rather than missing the file: rehash in zsh, hash -r in bash. A conda base environment can also keep its own bin ahead of the venv's, in which case conda deactivate before activating the venv. On Windows, run the activation command from PowerShell; Command Prompt uses .venv\Scripts\activate.bat instead.

See what the verifier catches

Run the short tour:

rgraph demo

It summarizes three real fixture checks in plain language and exits 0 when the clean handoff passes and both staged defects are caught. It calls no model and changes no study files.

rgraph demo runs three scenarios on a throwaway copy of the bundled example:

1. A clean run — nine gates pass.
2. A source identity is missing — evidence review stops the handoff and names the repair.
3. Data changed after the freeze — data_manifest.json rewritten after the protocol was frozen, digest and downstream references included, the way a re-run would. The stale chain invalidates T2, V1 and M1.

Scenarios 2 and 3 are the point. Neither can be prevented by a prompt; both are caught by a file digest.

Inspect one case in detail with rgraph demo --scenario 1, 2, or 3. The two detailed failure cases exit 1 because their selected gate fails on purpose. rgraph demo --all preserves the full three-screen diagnostic tour and also exits 1.

Start your own study, without editing JSON

The demo shows the verifier working on somebody else's evidence. This starts yours:

rgraph setup         # once per machine: detect tools and approve role assignments
rgraph doctor        # preflight PATH, login, assignment and capabilities
rgraph init          # answer a short study and governance wizard
rgraph decide        # choose the waiting human gate and read its two claims
rgraph status        # see exactly where the run stands and what to do next

rgraph init asks for the question, scope, constraints, success criteria, ethics applicability, data rules and responsible person in ordinary language. It previews the answers, then writes and seals meta.json, problem_spec.json and governance_record.json itself. Cancelling the preview writes nothing. From there, status always prints one recommended command and next walks the twelve units in graph order. A unit cannot run before its incoming gate passes; when the next graph node is a gate, next stops and points to the appropriate challenge, decide, or revise command instead of skipping it.

For a script or CI job, pass the same details as JSON or YAML with rgraph init --from study.yaml; study.example.yaml is a copy-ready template. If the setup of an existing run needs changing, rgraph init --edit updates only those three setup files and preserves the rest of the run. --force remains the explicit replace-everything option.

A human gate needs a human

Four of the nine gates are human gates, and check cannot decide one. It reads files, recomputes digests and compares strings — all of which it can do while nobody is watching. So a human gate stays AWAITING until somebody answers it:

GATE H1 / SCOPE, CONSTRAINTS & RESEARCH INTENT AWAITING
-------------------------------------------------

What this gate checked
  [PASS] Presence
  [PASS] Schema
  [PASS] Provenance
  [PASS] Staleness
  [FAIL] Decision
         no human decision recorded
  [PASS] Budget
  [----] Scientific correctness was not determined

  Run next:  rgraph decide H1

Everything mechanical passed. AWAITING is the screen saying so and stopping: it is not a problem report, and no amount of re-running turns it green.

rgraph decide offers a numbered gate menu when the gate ID is omitted, then asks what the gate declares it proves, one line at a time, and records the answers and who gave them. The questions are not invented: they are the proves entries already written in gates.yaml.

This gate proves 2 thing(s). It cannot prove them for you.

  1/2  Scope and constraints recorded
         Have you read problem_spec, governance_record and does this hold?
         [y] yes  [n] no  [s] stop > y

A no sends the gate back rather than opening it. Walking away records nothing. A pipe records nothing either: yes y | rgraph decide H1 would answer every question in order without anybody reading anything, so decide refuses a stdin that is not a terminal. rgraph setup offers --yes for exactly that situation; a human gate gets no such flag, because the answer is the point.

A decision is recorded against a self-declared name. decide and review default to your git config user.name and take --as "Your Name" when git has none or somebody else is answering. With neither, they exit 2 rather than file an anonymous decision. Both commands require a terminal. Even rgraph review --outcome release --as "Your Name" is refused through a pipe or non-interactive process; the flag can preselect a route for a person at a TTY. This prevents an accidental scripted approval path, but it is not identity authentication and cannot distinguish a person from software controlling a pseudo-terminal.

And because the attestation is pinned to the digests that were on the table, a later edit retires it — resealing repairs the hash, which is mechanical, but it cannot repair the reading, so the gate asks again. A retired gate reports STALE and sends you back to rgraph decide, not to rgraph revise: nothing upstream produced the change and no revision is owed for it.

A canonical SHA-256 is not something anyone types by hand, so rgraph seal computes it. Everything else the kit does rests on those digests being true, so editing a body and leaving its hash behind is itself a finding:

  problem_spec  BODY EDITED AFTER HASHING
        file no longer matches its content_hash: declared
        sha256:5f0e816734c7..., actual sha256:1a1e6c2f0434...
        Fix: re-run the unit that produces problem_spec, or `rgraph seal
             problem_spec` if you edited it on purpose

The four files

Separation of concerns is the backbone of the kit.

architecture.html is not a fifth contract authority. Its marked JavaScript ARTIFACTS and CONTRACTS block is generated from graph.yaml, gates.yaml and the repository schemas:

python scripts/generate_architecture_contracts.py

The generator replaces only that marked data block. SVG geometry, CSS, theme, responsive layout, prose and interaction code remain hand-maintained and are left byte-for-byte untouched. --check and the test suite reject stale output.

The same graph runs on anyone's combination of subscriptions. Provider identities, aliases, CLI detection, invocation templates, model suggestions and setup defaults all come from providers.yaml; adding a CLI or web provider does not require a Python change. The static public configurator is a release snapshot of the same registry, and its consistency is checked by the test suite.

Which model reads which role file is yours

architecture.html draws one pairing — Opus formulating, Sonnet implementing, Fable writing, a separate provider auditing — and rgraph setup opens with it. It is a recommendation, not a constraint. Press Enter to keep the complete proposal. To customize it, choose a role, provider, model and optional reasoning effort from numbered menus; internal provider/model syntax is not required. The final assignment is shown again before it is written.

The menus include every provider providers.yaml names and allow a custom model identifier when it is not in the suggested list. A provider whose capabilities cannot carry the selected role is omitted, unsupported effort levels are not offered, and the separation level is recomputed from the completed assignment. --yes skips customization and takes the proposal. A non-terminal caller must use --yes, so setup never hangs or writes an assignment it could not ask permission to write.

The model strings are the identifiers the CLIs answer to, which is not always the name the model is sold under: claude rejects sonnet-5 and takes claude-sonnet-5, and codex has no gpt-5.6 — only gpt-5.6-sol, -terra and -luna. Nothing here validates a model name; an unknown one fails at the provider, not at setup.

setup also reports CLIs it found on your PATH that providers.yaml says nothing about. It does not invent a call form for them — describe the CLI there and it becomes assignable, no code changes.

Three of the four describe the architecture and ship with the kit. assignment.yaml is yours, so it lives with you rather than with the install: rgraph setup writes ~/.config/rgraph/assignment.yaml once, and every study on the machine uses it. A study that needs a different pair of providers gets its own copy with rgraph setup --here, which wins over the machine default for that directory.

The verifier: two layers

Layer 1 — static, no run required

rgraph check --static

Layer 2 — dynamic, over a run/ directory

Existence, JSON Schema, provenance (produced_by, inputs[] upstream hashes, content_hash), staleness, reviewer separation, and the revision budget.

The last two are the kit's reason to exist. Neither can be prevented by instructions, and both fall out of a file digest:

• Staleness — every artifact carries the digest of everything it declares — who produced it, what it consumed, and the body — and the content_hash of each input it consumed. Both are recomputed on every read, so a file edited after a gate passed invalidates that gate and everything downstream, whether or not whoever edited it updated the hashes. Rewriting produced_by or dropping an inputs[] entry is an edit like any other.
• Reviewer separation — a challenge gate records who decided it and who produced what was decided on. A recorded run also binds the decision to the exact CLI argv, prompt digest, response-log digest, exit code and current artifact hashes. check verifies those bindings but never creates them.
• Unit execution provenance — every new provider call writes a host receipt containing its assignment, argv, unique log digest, exit code and current input/output hashes. A rejected invocation remains visible and cannot advance status merely because it left files on disk.

Reviewer separation, not independence

"Independent" promises more than a separate session or provider can establish. The CLI therefore reports a measurable reviewer-separation level instead:

CONTEXT ONLY always prints with its caveat:

Review separation
  Level : CONTEXT ONLY
  Note  : Reviewer uses a separate session, but the same model
          and provider. Correlated errors may remain.

rgraph never hides which level was actually achieved; it writes it into the release manifest. tests/test_separation.py holds the CLI to it. No level guarantees epistemic or statistical independence; correlated errors can remain across sessions, models and providers. The reference diagram uses the same "review role" and "separate audit" language rather than turning a design intention into a measured claim.

The honesty limit

The reviewer_id != producer_id and captured-command checks are not cryptographic provider attestation. It is a discipline mechanism. A determined user can write whatever identity they like — and would only be deceiving themselves. The kit is built to make the honest path the easy one, not to make the dishonest path impossible.

What it does not do

research-graph does not judge scientific correctness. It establishes that every artifact behind a claim is registered, versioned and traceable to its inputs. Every gate screen repeats this, in the same words, whether the gate passed or failed:

  [----] Scientific correctness was not determined

It does not read the manuscript prose either. M1 checks that every claim in claim_evidence_map maps to a computed result or a snapshotted source, and that the manuscript and the map name the same claims. It does not parse the text, so a number that appears only in a sentence — quoted from the raw data but never registered as an estimate — is outside what any gate can see. The example run does exactly this: it reports 8.99 dB at +10 dB SNR, which is true of raw_results.jsonl and absent from statistical_report.json, because the frozen protocol registered three low-SNR points and adding a fourth after the freeze is the thing this kit exists to prevent. The claim built on it is marked extrapolation, which is the honest handling — but it is the human who has to notice, not the verifier.

Also deliberately absent: no scheduler or continuous orchestrator, no model API client, no multi-provider abstraction layer, no database and no remote server. Explicit execution commands call the configured local subscription CLI once and return control. rgraph ui is a loopback-only application over the same local files and the same Python checks: it can start one approved provider process in the background and watch it, but it schedules nothing, runs no shell, and adds no provider of its own.

The local application

rgraph ui binds to loopback only and refuses any other host. Every request that changes something, and every read of an execution log, carries a per-session token in a request header — never in a URL, and never written to a log. There is no /api/decide and no /api/review: a human gate cannot be recorded from a browser at all.

An approved plan runs as a child process with shell=False and an argument list built from providers.yaml. Nothing typed in the browser reaches a command line, there is no stdin forwarding, and there is no general terminal. A provider that needs an interactive login is reported by rgraph doctor and sent to the terminal rather than wrapped.

Reading it back afterwards. Each execution leaves three files under the study's logs/jobs/, named for when and what they ran (20260807T0830-u01-419e1bb): a JSON record the browser reads, an event log for reconnecting, and a Markdown transcript for a person — what ran, what it read, which files it changed, the timestamped output, and how it ended. From the terminal, rgraph jobs lists them and rgraph jobs u01 opens the last one.

Where the logs are, and who can read them. The complete provider log stays where it always was, under the study's logs/ directory, and it is not redacted — anyone with access to this computer can read it. What the browser shows is a separate, bounded copy with control sequences stripped and known credential shapes replaced. That reduces exposure; it does not establish that provider output contains no secret. Operational job records live beside the logs under logs/jobs/; they are not research artifacts, carry no schema, and are never read as evidence.

Process completion is not artifact success. The console reports the process state and the artifact validation separately: the process exited, the run is readable, the declared outputs are present and schema-valid, their digests recompute, the producer identity matches, and a receipt was written. A provider that exits 0 and writes nothing fails at the second stage, and says so.

Running it: four tiers

Tier 3 is a design intention, not a feature: providers.yaml has no API-backed provider kind and rgraph contains no model HTTP client. Tiers 1–2 use the installed subscription CLIs. rgraph challenge invokes exactly one assigned CLI and binds its prompt and response log to the gate record; it is not an autonomous orchestration loop. A web-only reviewer can be used manually, but the public-beta CLI does not mislabel a pasted response as a verified invocation.

Tier 2 is the kit's most distinctive configuration: real cross-provider auditing for the price of two subscriptions and no API spend. These call forms were run against the installed CLIs on 2026-08-02 and are what providers.yaml records:

codex exec -c model="gpt-5.6-luna" - < roles/reviewer.md   # codex-cli 0.144.6
claude -p --model claude-opus-5 < roles/planning.md        # Claude Code 2.1.220
codex exec -p fugu -c model="fugu" - < roles/reviewer.md   # Sakana Fugu, via codex

What has not happened is an end-to-end study driven this way. example-run/ is a fixture with authored provenance, template-run/ is empty, and no run in this repository was produced by the identities it names. The verifier is exercised; the multi-agent loop around it is not yet evidenced.

A subscription is not an API key

Subscription CLIs carry rate limits, and orchestration is not automatic: you move between steps yourself. Full automation is tier 3.

Commands

check only verifies. decide records a terminal human attestation; the local UI displays the same gate state and points to rgraph decide or rgraph review, but cannot record either decision. challenge launches the assigned reviewer once and writes a record only after the structured response, current input hashes and captured log validate. This split prevents check from inventing either kind of decision.

review is the last of those decisions and has two shapes. release, null-result and stop close the run and write a release manifest; after one of them, status and next report the run closed rather than proposing more work. revise and narrow spend one of FINAL's revisions and route the work back to a unit instead, writing no manifest. Only release exits 0.

Global flags (--run, --root, --verbose, --no-banner) are accepted on either side of the command name.

rgraph next shows exactly what it would run, prints No command has been executed., and offers Execute, Dry run and Stop as numbered choices. It runs one subprocess and returns control. --unit cannot bypass an unresolved upstream gate. rgraph challenge has the same one-subprocess boundary for a reviewer. There is no scheduler and no loop.

The example run

example-run/ is a fixture, and it says so: its meta.json carries "provenance": "synthetic", and every rgraph screen that reads it prints that before anything else. The experiment, the statistics, the DOIs and the hash chain are real; the produced_by.identity fields are illustrative, because no provider was ever invoked and no reviewer ever decided a gate. example-run/README.md lists exactly what is real and what is not. A kit about honest provenance does not get to be vague about its own.

It answers a real question — does a learned channel estimator actually beat LMMSE at low SNR? — over twenty seed-matched replications with a protocol frozen before execution.

The estimator it labels learned is not a neural network. It is a tuned delay-domain filter: the structure a learned estimator recovers in this setting, written out explicitly so the benchmark stays deterministic and reviewable. The result therefore bounds the value of that structure, not of any architecture. The code says so, the manuscript says so in its Method and Limitations, and it is said here too, because a reader who stops at this README should not leave with the wrong picture.

All four DOIs in its corpus were resolved against Crossref and verified by direct lookup. None was hand-written. If they had been, the kit's own E1 gate would fail on the kit's own example.

The run reports an honest negative. Registered hypothesis h-02 said the advantage would be largest at the lowest SNR. It is not: 2.09 dB at -10 dB against 8.99 dB at +10 dB. The manuscript says so, the verification report grades it major, and the claim–evidence map marks the trend claim as an extrapolation. That is what the kit is for.

Credit and neighbours

codejunkie99/graph-engineering defined the term and wrote its textbook: knowledge graphs are what agents remember, task graphs are what agents run. Its four rules — delete fake edges, verify at every stage, separate verifier contexts, keep a stop rule and a human gate — are the rules this repository turns into code. They wrote the textbook; we wrote the compiler. Complementary, not competing.

See also Arbor (arXiv 2606.11926), which solves a different problem: finding a better result through autonomous optimisation. Arbor runs; research-graph audits.

Four decisions that departed from the original design

The kit was specified before it was written, and four points came out other than planned. The specification is not shipped — it described an eight-command tool with a different integrity model, and keeping a stale one next to a live verifier is the failure this kit exists to catch. It stays in the history at v0.1.0. What changed is worth recording, so it is recorded here:

1. 21 artifact schemas, not 18. Every gate input has a schema, including the retrieval unit's kg_snapshot, so the reachability lint has no blind spot. The plate remains hand-designed, but its artifact and gate data is generated from the executed configuration. Freshness and semantic tests cover gate contracts, artifact ownership and typed revision reasons; no undeclared diagram/configuration divergence is allowed.
2. Graph nodes are the 12 work units, not the 5 pipeline stages. The CLI output requires unit granularity; each unit carries a stage field and rgraph status aggregates the five-stage row from it.
3. Run artifacts are JSON; only the four config files are YAML. The kit ships a deliberately small YAML-subset parser (no PyYAML dependency) and that parser should not be the one reading your evidence.
4. Gate decisions are gate records, not an artifact. They live in run/gates/<GATE>.json under their own schema and are not among the 21.

Requirements

Python ≥ 3.11, tested on 3.11, 3.12 and 3.13 across Linux, macOS and Windows. Installing with uv covers that requirement for you; a checkout install expects you to have the interpreter already. Three direct runtime dependencies: jsonschema, referencing and rich.

The verifier is offline-first. rgraph check E1 --online is the optional network exception inside validation, which resolves each DOI against doi.org; without a network it reports which DOIs it could not reach and exits 1 because the requested check is incomplete, rather than calling them fabricated.

Agent execution is separate from validation: next --execute, challenge, and doctor --probe-models explicitly contact the configured provider through its local CLI. No other command silently starts a provider.

The graph, gates, schemas, role contracts and both example runs ship inside a wheel. A wheel install uses that packaged copy outside a checkout; an editable install resolves the source checkout's copy from any working directory.

The distribution is published on PyPI as rgraph; the product, repository and Python module remain research-graph, huguryildiz/research-graph and rgraph. Releases use PyPI Trusted Publishing, so no long-lived upload token is stored. A tag-pinned GitHub install remains available when an exact source reference is preferred:

uv tool install "git+https://github.com/huguryildiz/research-graph@v0.4.1"

To remove it: uv tool uninstall rgraph, or pip uninstall rgraph from a checkout, then delete your run/ directory and ~/.config/rgraph/assignment.yaml if you want the state gone too.

Contributing

uv sync --frozen --extra dev
uv run --frozen --extra dev pytest -q

Details, including what CI checks and why, are in CONTRIBUTING.md. What counts as a security issue — and the two limits that mean most reports are not one — is in SECURITY.md. Release notes live in CHANGELOG.md.

Licence

MIT — see LICENSE.