HypGraph

Don't believe the hype. Test the hyp.
The evidence layer for AI research agents.
hypgraph.com · for agents · for engineers · record format

Agents doing research write fluent summaries. Some of the numbers in them match no result. A clean red-team run of 50 prompts becomes "the model never leaks". A search too weak to find anything becomes "ruled out", and three copies of one result get counted as three lines of evidence. Then other claims, decisions and deployments get built on top. HypGraph is the record agents write into instead. It checks every claim as it's written, does the arithmetic the writer skipped, and refuses:

$ hypgraph add claim statement="The model never leaks training data." exposure=1000000 exposure_unit=users …
refused: claim refused:
- 'never' rests on 0 of 50: that only rules out it happening more than 5.8% of the time (95% upper bound).
  Honest version: 'no occurrences in 50 trials (95% upper bound 5.8%)'. To support a rate below 1 in 1,000 you need
  about 2,995 trials without one. Across the 1,000,000 users this claim applies to, that leaves up to 58,155 not
  ruled out. [ABSOLUTE_UNDERPOWERED]

The honest version goes in, and it carries its consequence wherever it's read:

active claim · unchecked · 1 independent line of support · … · 0 of 50 = at most 5.8% at 95%, up to 58,155 of 1,000,000 not ruled out

The arithmetic of never

"Never" is the most expensive word in research, because everything built on it inherits it. Zero failures in n trials doesn't mean a rate of zero. It means a rate below 1 − 0.05^(1/n), about 3/n, at 95% confidence. HypGraph holds every claim to that:

Every refusal has a stable code, the arithmetic, and the sentence that would be honest, so an agent can repair a claim in one step. hypgraph bound 0 50 --exposure 1e6 (or the MCP tool bound) answers "what does my count actually support?" before anyone writes the word.

Every refusal is also logged: who tried to say what, which rules caught it, and whether they came back with the honest version.

$ hypgraph refusals
## Overclaims caught — 4 refused, 3 came back honest
| author  | refused | fixed | open | most common |
| analyst | 4       | 3     | 1    | EXCLUSION_WITHOUT_POWER ×1, ABSOLUTE_UNDERPOWERED ×1, UNBOUND_NUMBER ×1 |
### Latest
- analyst: “Prompt format makes no difference to accuracy.” — EXCLUSION_WITHOUT_POWER → not fixed
- analyst: “The model never leaks training data.” — ABSOLUTE_UNDERPOWERED → fixed as CL2
- analyst: “The fine-tune is 5 points better than the baseline.” — UNBOUND_NUMBER → fixed as CL1
- analyst: “The fine-tune always beats the baseline.” — ABSOLUTE_WITHOUT_COUNT, COMPARATIVE_WITHOUT_COMPARISON → fixed as CL1

And because most hype lives in the write-up, not the record, hypgraph check summary.md lints the final report against the record. It catches numbers no result contains, claims called "established" that the record has as unchecked or contested, citations to withdrawn claims, and "never" / "no effect" / "significantly" with nothing behind them. It exits 1 on errors, so it can gate CI:

$ hypgraph check summary.md
summary.md:8: error REPORT_OVERSTATES_STANDING: 'established' — but CL2 is unchecked in the record …
summary.md:9: warning REPORT_UNSOURCED_NUMBER: 700 isn't in any result, measurement, claim, scope or constant …
summary.md:13: error REPORT_UNKNOWN_CODE: CL9 isn't in the record for this investigation — a citation to nothing.

And when something does fall, the record says what falls with it:

$ hypgraph withdraw CL1 --why "the held-out set leaked into training"
CL1 — withdrawn claim · …
  ⚠ 2 nodes rest on CL1: 1 decision (D1); 1 claim (CL2) — now AT RISK

Derived, never asserted

Nothing in the record is ever set as known. How well each claim is known is derived on every read:

• reviewed, and which version? Only named reviewers count, and never on their own work. A revised claim goes back to unchecked.
• does it rest on something that fell? Withdraw a premise and everything built on it is flagged AT RISK. A decision whose evidence changed is flagged REVISIT.
• how many independent lines? Results that share a run, a data file or a source count once.
• how strong is the best one? From an agent's report, through a run your job runner recorded and a separate implementation, up to a replication on new data.

Every write is an event in a SHA-256 hash chain. A record can be handed to a reviewer, a regulator or another team, and they can verify it without trusting you.

One record, five disciplines

The rules above are the scientific case written as code. A profile says what a domain means by evidence, acceptance and done — and, from 0.5, carries the discipline's own workflow: the stages a question passes through, the gate out of each, what invalidates evidence, the typed shapes. The record enforces all of it, so none of it depends on a prompt being followed. Pin one on the investigation ("profile": "software@2"); without one, the record behaves exactly as before.

The stage is derived, never set: g.stage(question) reads it off the record (the first stage whose gate is not met, every gate MET / NOT MET with what would meet it); GET /api/graph/stages; MCP tool stage. A workflow widens what puts dependants AT RISK — uses (a corrected data file, a superseded diff or instance), cites (a withdrawn source) — and evidence goes stale when the artifact it used is superseded or its valid_until has passed. A result whose run names no file it read goes stale when an input file of its question changes after it: nothing shows it read the new version (name the files with uses and only those count). An honest "it can't be answered" (a checked inconclusive claim), a declined thesis (no_position) and a verified failure each complete a question the gates alone never would.

Profiles are versioned and frozen: a record written under trading@1 means the same thing in a year, and the @1 profiles stay exactly as they were. completion(inv) reports each criterion; hypgraph serve exposes /api/graph/completion and /api/graph/profiles. The method each workflow is taken from is quoted, one page each, in docs/workflows/. Adding a discipline is one Profile with a Workflow.

Connect your agent

pip install "hypgraph[mcp]"
claude mcp add hypgraph -- hypgraph --db study.hypgraph --as claude mcp     # any MCP client works
hypgraph skill --install                                                     # + the Agent Skill (.claude/skills/hypgraph/SKILL.md)

The server writes as one author, fixed when it starts. The agent can't pass another name or write as your job runner, and its verdicts count only if you name it as a reviewer. Tools: overview, find, show, start_investigation, add, link, revise, withdraw, retract_link, review, next_experiments, work_queue, claim_work, register_run, bound, check_report, refusal_log, report, verify, schema. The method is in AGENTS.md; the short version is llms.txt.

Wire it into your system

The core is one Python package with no dependencies. You decide who is trusted to do what:

from hypgraph import Graph, Policy

g = Graph("research.hypgraph",
          policy=Policy(reviewers={"critic", "stats"},      # whose verdicts count
                        instrument="runner"),               # the only author that files lab-recorded runs
          spend=my_ledger.spend)                            # optional: prices the work behind each investigation

with g.writing_for(task_id):                                # every event carries the task, for cost tracing
    g.create(author=agent_name, type="claim", inv=inv, fields={...}, links=[...])

• Multi-agent: give each agent its own author name. Make your critic agents reviewers. Have your job runner write as the instrument, so its runs count as recorded, not reported.
• Your web framework: hypgraph.server.dispatch(graph, method, path, query, body) is the JSON API, and hypgraph.ui.static_dir() is the HypGraph UI to mount next to it.
• Watching it live: GET /api/graph/stream?inv= pushes every event as it is written (Server-Sent Events, resumable with Last-Event-ID), including writes from another process on the same file, such as a swarm running against the record. GET /api/graph/profile?inv= gives the investigation's profile as a UI speaks it: each node type's label ("Requirement", "Thesis"), the workflow's stages and the report's name. A swarm's telemetry sidecar (<db>.telemetry.jsonl, written by hypswarm) rides the same stream as telemetry events, so the Live view shows every model and tool call as it happens.
• CI: hypgraph export -o record.bundle.json && hypgraph verify record.bundle.json. The bundle is a single file your reviewers or auditors can check with SHA-256, in any language or in the browser. --telemetry carries the swarm's sidecar too, outside the chain, so the bundle replays the run with no server.

The full guide is at hypgraph.com/engineers.

Try it

pip install hypgraph
hypgraph demo        # a complete worked investigation (Broad Street, 1854) built into ./research.hypgraph
hypgraph serve       # explore it in the HypGraph UI: http://127.0.0.1:8765
hypgraph report      # the facts, generated from the record

Or watch a swarm work in each profile, replayed live from its record: hypgraph.com/examples. Or try to get a claim past the real engine in your browser: hypgraph.com/play.

Where it came from

HypGraph began as the research graph of a team of AI agents running long investigations under governed egress, and was pulled out so anyone can use it. Each rule exists because the agents made the mistake behind it:

• a validation that was blind to the answer;
• a null model that broke the system's own rules;
• a summary whose numbers matched no result;
• "never" written about something that happened once in a thousand.

That team still runs on it, and its graph UI is built from this package. Against the team's live record (1,357 events, 790 nodes), the library derives the same standing for every node as the engine it replaced.

Performance

The engine is pure Python and stays fast at the sizes research records reach. On dev/bench.py's record (one investigation of questions, results, claims and reviews), at 10,001 nodes deriving every node's standing takes 0.34 s on a Windows laptop, reading one node 0.07 ms, and a write about 12 ms, mostly SQLite committing to disk; derivation grows in step with the record. A write costs what it touches: cached standing is kept for every node the write can't affect, and a randomized suite checks that against a full rebuild. Run python dev/bench.py for your machine.

Made by

HypGraph is made by Nicolas Houghtaling at VoodooMods LLC.

License

Code: Apache-2.0. The record format (docs/spec.md) and the docs: CC BY 4.0. Cite it with CITATION.cff.