Clew

Clew works out what has to happen downstream when something upstream goes bad. A buggy tool, a contaminated sample, a reference update, a withdrawn consent. It rebuilds the lineage of your Nextflow runs and tells you exactly what to delete, re-run, or disclose, with a plan you can hand to an auditor.

A clew is the ball of thread Ariadne gave Theseus. You follow it back out.

The problem

Something upstream is invalidated after the fact. It was already aligned, recalibrated, aggregated into a cohort report, maybe cited in a paper. Nobody can mechanically answer "what do I now have to delete, re-run, or disclose?" In practice a person answers it with a spreadsheet, or it is not answered at all.

Provenance tools record where data came from. None of them record whether a contribution can be taken back out. That is the difference between a history and a recall plan, and it is the gap Clew fills. Every derivation edge gets a contribution class:

Anything unknown fails closed to IRREDUCIBLE. Telling someone their data is clean when it is not is the one error that ends up in front of a regulator.

The headline, first

One withdrawal, two pipelines. A yeast sample was withdrawn; its rnaseq run had published a count matrix; a separate differentialabundance run consumed it. Clew stitches the two runs' graphs at that published file and answers across the boundary:

TRIGGER: withdrawal of SRR10441036_cox4d
AFFECTED: 57 of 183 tasks        (46 in the rnaseq run, 11 in the DE run)

da:29/ae3d99  DESEQ2_DIFFERENTIAL   REGENERABLE  shared
    via rna:f2/cefd0f[STAR_ALIGN] -> rna:0c/8143cf[SALMON_QUANT]
     -> rna:c9/9a30ba[CUSTOM_TX2GENE] -> rna:8e/b5be55[TXIMETA_TXIMPORT]
     -> da:e8/91c345[VALIDATOR] -> da:29/ae3d99[DESEQ2_DIFFERENTIAL]

The evidence chain starts at the sample's alignment in one Nextflow launch and ends inside another, crossing at a checkable published path. Engine lineage sees each run in isolation; this graph is the part nobody else has.

clew stitch --graph rna=clew/data/graph_rna.json --results rna=<rnaseq results dir> \
    --graph da=clew/data/graph_da.json --out graph_chain.json

clew impact --pipeline rnaseq --graph clew/data/graph_chain.json \
    --samplesheet clew/data/samplesheets/rnaseq_yeast.csv --donor SRR10441036_cox4d

See it in two minutes

pip install clew-lineage

clew demo

It answers three questions, one per audience, then crosses a run boundary — all from the same engine, over a real graph that ships with the package: an nf-core/sarek run with 5 synthetic donors, 81 tasks, 344 edges.

The base install has no dependencies beyond Python 3.11 — the demo, the extractors and clew impact are stdlib-only and stay that way. Only the event log needs a database driver: pip install 'clew-lineage[log]'.

Prefer not to install anything? Clone the repo and run python3 -m clew demo from its root; every command below also works as python3 -m clew ....

Use it on your own run

Clew computes over a lineage graph. Two extractors produce one today, and they emit the same JSON, so everything downstream is identical either way.

Preferred: Nextflow's native data lineage (25.04+). Enable it in your Nextflow configuration before the run, exactly as the Nextflow docs describe:

lineage {
    enabled = true
}

Nextflow then records every task, output file, and link into a .lineage store with content-addressed lid:// identifiers. Clew has no opinion on where you put that setting; it only reads the store the engine writes. Build the graph from it:

clew extract-store --store /path/to/.lineage --list-runs

clew extract-store --store /path/to/.lineage \
    --run <run-name> --json-out graph.json

The engine is the best witness of what it ran: inputs are typed, external files carry checksums, and every task names its run, so sharing one store across many runs is safe by construction.

Also supported: Workflow Run RO-Crate, as written by the nf-prov plugin. Labs that already publish crates for journals or archives have lineage on disk without knowing it:

clew extract-crate --crate ro-crate-metadata.json --json-out graph.json

A crate records what ran, not how to re-run it: no script, no workdir. So tasks from a crate classify as IRREDUCIBLE and their storage reads DESTROYED unless published copies are mapped — fail-closed, by design. Prefer the lineage store when both exist.

Fallback: the work/ symlink extractor, for runs that already happened without lineage enabled. Nextflow stages inputs as symlinks to save disk, and those symlinks accidentally record the entire history of the run. No pipeline modification, works on any Nextflow version:

clew extract-work \
    --jsonl /path/to/weblog/<run-id>.jsonl \
    --work  /path/to/work \
    --json-out graph.json

Do this during or right after the run. nextflow clean removes the symlinks, and lineage that was never captured cannot be reconstructed.

On the same sarek pipeline the store and symlink extractors produce identical impact numbers, and the test suite enforces that equivalence.

Capture and computation stay separate on purpose. The engines are getting good at recording what happened. Clew's job starts where they stop: whether a contribution can be taken back out, and what must happen when it cannot.

Anatomy of a trigger: selector × mode

Clew has no hardcoded scenarios. Every trigger is the combination of two independent choices, and the familiar stories are just named cells in that grid.

The selector answers "where does the problem enter the graph?" Three ship today:

The mode answers "what kind of wrong is it?" Two exist, and they are not interchangeable:

• remove — the source must be taken out (a withdrawal). Ownership matters: an artifact existing only because of this subject has nothing left to serve, so it can be destroyed.
• distrust — the data is suspect but still wanted (contamination, a tool defect, a stale reference). Nothing is destroyed; the worst verdict is quarantine, because you will want these artifacts again once the cause is fixed.

The stories, mapped:

Defaults preserve the common cases (--donor implies remove, the others imply distrust), and --mode overrides them: contamination is --donor X --mode distrust.

Will selectors be extended? Yes — that is the extension point. A selector is anything that can name a set of entry nodes; the engine only ever sees the set. Obvious future selectors: by file checksum (one exact artifact), by batch or time window (every task in the run of a bad reagent lot), by facility, by edge kind (everything calibrated against a control, not merely derived from it). Modes are the closed part: new verdicts would change what remediation means, so a new mode is a design event, not a plugin.

Three questions it answers

Pipeline engineer: "We bumped the reference genome. What must be re-run?"

clew impact --graph clew/data/graph5.json --samplesheet clew/data/donors.csv --input genome.fasta

On the sample run: genome.fasta was consumed directly by 41 tasks, and 72 of 81 tasks are calibrated against it. The other 9 are provably out of scope, with the derivation chain printed as evidence for every claim.

QA: "A defect was reported in a GATK4 container. What did it produce?"

clew impact --graph clew/data/graph5.json --samplesheet clew/data/donors.csv --container gatk4

On the sample run: 16 tasks ran the container, 68 of 81 tasks are suspect. Note that nothing is destroyed. A defect casts doubt, it does not remove a source, so artifacts are rebuilt rather than deleted.

Compliance: "A donor withdrew consent. What happens now?"

clew impact --graph clew/data/graph5.json --samplesheet clew/data/donors.csv \
    --donor donor_003 --assertions clew/data/assertions.json

On the sample run: 16 of 81 tasks are affected. The 15 that exist only because of this donor are destroyed, where the artifacts are still on disk. Add --work-root pointing at the run's work directory to get those verdicts; without it the storage-dependent ones come back UNDETERMINED, which is the honest answer rather than a guess. The cohort report that also serves the other donors, and was cited in a publication, resolves to NOTIFY_ONLY instead: you cannot unpublish, so the answer there is disclosure, not deletion. One traversal, two verdicts, which is the whole point.

Publication is an external assertion, not something Clew infers. The assertions file records who claimed it and when:

{
  "published": [
    {
      "task": "c9/023b13",
      "what": "cohort MultiQC report, cited as Supplementary Fig 1",
      "asserted_by": "name@example.org",
      "date": "2026-08-21",
      "reference": "doi:10.0000/example"
    }
  ]
}

Storage is checked, never assumed

Every verdict above depends on two different kinds of fact, and they are not the same kind at all:

• Lineage — what was derived from what. Permanently true, and it is what the graph holds.
• Storage — whether the bytes are still there. True only at the instant you look, and not in the graph at all.

Whoever asks Clew a question is often not standing where the pipeline ran: a different host, a CI runner, a laptop reading a graph someone emailed over. So Clew does not check unless you tell it where to look:

clew impact --graph graph.json --samplesheet samplesheet.csv \
    --donor donor_003 --work-root /path/to/work

Without --work-root, storage is unverified and any verdict that depends on it is reported UNDETERMINED rather than guessed. Verdicts that hold whatever the disk says are still returned — under v2 a published artifact is NOTIFY_ONLY either way, and that is an answer, not a guess:

  UNDETERMINED  (57)  — no verdict; see below
    storage state not verified, and the verdict depends on it. Verifying
    would decide between ALREADY_GONE, DESTROY or QUARANTINE. Refusing to
    guess: assuming the artifact survives over-claims work, and assuming it
    is gone reports an obligation as already discharged.

With it, Clew looks and says what it found — on the cross-run graph, 46 tasks whose scratch really was cleaned, and 11 whose artifacts are still there:

AFFECTED: 57 of 183 tasks
  ALREADY_GONE  (46)  — no longer exists; nothing to do
  REGENERATE    (11)  — recompute from the remaining sources

ALREADY_GONE is now only ever reached by looking and not finding. It used to be returned whenever the recorded path failed to resolve — which fired identically when the volume was not mounted, when the graph came from another machine, when the extractor never recorded a workdir at all (every RO-Crate task), and when the published fixtures had their paths anonymised. All of those produced "no longer exists; nothing to do", silencing exactly the artifacts that carry obligations. A false negative that discharges an obligation is worth far more care than a false positive that wastes work.

Recorded paths belong to whichever machine ran the pipeline, so only the two-character prefix and task hash are joined onto the root you supply. A graph stays portable between hosts without pretending its absolute paths mean anything locally.

An undetermined item is not clean, it is unanswered — it carries the set of verdicts still in play, so a CI gate can fail on it and a reader can see exactly what checking the disk would settle.

The event log

Everything above is a computation over facts. If the facts can be edited afterwards, none of it is worth anything — so the facts get their own store, on Postgres, whose only job is to make revision impossible for the application and detectable for everyone else.

pip install 'psycopg[binary]'

clew log --dsn "$CLEW_ADMIN_DSN" init \
    --writer-password "$W" --auditor-password "$A"

clew log --dsn "$CLEW_DSN" append --type ContainerDefectReported \
    --subject "gatk4:4.2.1" --actor qa.lead@example.org \
    --effective-from 2026-08-10T00:00:00+00:00 \
    --body '{"defect":"BQSR miscalibration","reference":"JIRA QA-4471"}'

Two clocks, deliberately. effective_from is when the fact became true; recorded_at is when we learned it. A defect discovered in August was true in March, and the release made in between was made in good faith and still has to be disclosed. One timestamp cannot say that, and adding the second later means re-interpreting every historical row.

The log has a clock; the computation does not. A plan stays a pure function of the facts and byte-identical on replay. Learning is dated, deciding is not.

Three layers, each stopping what the next cannot. They are not redundant:

The writer role holds SELECT and INSERT and was never granted UPDATE, DELETE or TRUNCATE; a role cannot grant itself a privilege it does not hold. This is why the log is on a server rather than in a file — whoever holds a file holds every privilege over it. TRUNCATE has its own statement-level trigger because it fires no row triggers at all and would otherwise empty the log in one statement.

Verification needs no database and no driver:

clew log --dsn "$CLEW_AUDIT_DSN" verify

FAILED at seq 2
  content does not match its own hash: this entry was edited after it was written
  1 entries verified before the break

What it does not prove. The chain detects editing. It does not detect a truncated tail — a shorter chain is still self-consistent — nor a full rewrite by someone holding the owner's credentials. No hash chain closes that alone; anchoring the head hash outside the database does, which is what the evidence bundle is for. Owner and application must be different identities and the owner's credentials must stay out of CI. Clew cannot enforce that from inside, and says so rather than implying otherwise. Both limits are asserted as passing tests, so nobody reads ok as "nothing was lost".

Policy versioning

Every verdict above is a verdict under a table. If that table is an if-ladder in Python, editing it silently re-interprets every plan ever produced: a plan from March says QUARANTINE, the code says QUARANTINE today, and nobody can tell whether it said QUARANTINE in March. The history is unfalsifiable, which is the same as worthless.

So the table is data, with a version and a content hash:

clew rulebook show

  R3   exclusive=True, storage=WRITABLE                     -> DESTROY
       Exists only because of this subject and the bytes can be changed.
       Nothing else needs it, so it goes entirely.

Every plan names the policy in its header and every line cites the rule that decided it, so "policy v1, rule R5, these hashes, re-run and get the same answer" is a checkable sentence.

Rule order is semantics — first match wins, and an omitted dimension is a wildcard. That is not a detail: it is the entire difference between the two shipped versions.

clew rulebook diff v1 v2

order  v1: R1 R2 R3 R4 R5 R6 R7 R8
       v2: R2 R1 R3 R4 R5 R6 R7 R8

  R2  position 2 -> 1
      - Immutable history — published, or already past a trust boundary.
        Terminates remediation, not notification: you cannot unpublish...
      + Immutable history — published, or already past a trust boundary.
        Asked first, before existence: destroying our copy does not reach
        the published or transferred one, so the obligation to disclose
        survives the bytes.

v1 asked "does it still exist?" before "was it published?", so a published artifact whose working copy had been deleted came back ALREADY_GONE — nothing to do. Deleting your copy of something does not un-publish it.

The practical consequence was sharper than it first looks. R1 is the only rule that can yield ALREADY_GONE, so putting it first made every verdict depend on the storage state — under v1, nothing at all is decidable without a disk check. Under v2 a published artifact resolves without one, because the answer genuinely does not depend on it:

clew impact --graph clew/data/graph5.json --samplesheet clew/data/donors.csv \
    --donor donor_003 --assertions clew/data/assertions.json --policy v1

POLICY: v1  dbb59de6d85fc0f8       POLICY: v2  e6ba60ffe6763949
  UNDETERMINED  (16)                 NOTIFY_ONLY   (1)   c9/023b13  MULTIQC
    c9/023b13  MULTIQC               UNDETERMINED (15)

v1 is still here, byte for byte. --policy v1 resolves it, and a plan computed in January replays under the table that produced it rather than under today's. A semantic change is a new version, never an edit — the shipped hashes are frozen as literals in the test suite, so editing one fails the build and says to add a version instead.

Adoption is a logged fact. clew rulebook register writes a PolicyAdopted event carrying the whole table, not a pointer to it — a pointer to code is worthless six months and four releases later:

clew rulebook register --dsn "$CLEW_DSN" --actor qa.lead@example.org

Validation refuses, it does not warn. A policy naming an unknown action, an impossible value, a duplicate rule id, or a rule with no rationale is rejected at load. The one that matters most is a mistyped dimension, which would otherwise load cleanly and silently never match — and a rule that never matches is indistinguishable from a deleted one, except that the file still shows it and everyone believes it applies.

Two guards sit outside the rule list where no policy can reach them: an unrecognised contribution class becomes IRREDUCIBLE before matching, and falling off the end of the rules yields QUARANTINE rather than an error or a pass.

Be precise about what that guarantees. It fixes the facts, not the verdict. A policy mapping IRREDUCIBLE to PURGE is expressible, would be wrong, and Clew will run it. That is not a hole — it is the reason the table is data. Wrong logic in an if-ladder is invisible in a code review nobody does; wrong logic in a hashed, versioned, rationale-carrying file sits in the open with a rule id on it.

This is the core table, not the customer's policy. It defines what the classes mean, so changing it changes the semantics of every historical plan — which is exactly why it is versioned. Which of a customer's events map to which class, what counts as published, what a given withdrawal tier may reach: that is a separate object, it lives in domains/, and it is not this file.

Evidence bundles

A plan on someone's terminal is a claim. A bundle is the artifact that lets a third party check it without trusting you, without your database, and without your code being the thing that says so.

clew evidence build --out bundle/ --plan plan.json --dsn "$CLEW_DSN" \
    --input graph.json --input samplesheet.csv --seal-into-log \
    --actor qa.lead@example.org

clew evidence verify bundle/

verify reads a directory. No database, no network, no credentials, no driver installed — an assessor who does not trust the party that produced a bundle must be able to check it anyway, and any step routing through the producer's infrastructure defeats that.

  ok   files      6 files, all hashes match
  ok   log        2 entries re-chain to the recorded head (seq 2)
  ok   policy     v2 matches the hash the plan cites
  ok   replay     all 57 verdicts recompute identically from the bundled facts and policy
  ok   signature  sealed by qa.lead@example.org

replay is the one that matters. A folder of documents proves only that somebody assembled a folder. Replay re-derives every verdict from the bundled facts and the bundled table, offline. Rebuild the manifest so all the hashes match again and change only the conclusion, and it still fails:

  ok   files      6 files, all hashes match
  FAIL replay     1 of 57 verdicts do not reproduce:
                  da:06/31c01f: recorded ALREADY_GONE, recomputes to REGENERATE

Bundles are clock-free. The same inputs produce the same bundle hash, which is testable and tested. A timestamp inside would change the hash on every build and quietly destroy the reproducibility claim; time lives in the log, which is the thing with clocks, and sealing is itself a logged event.

This is what closes the log's open gap

A hash chain detects editing but not truncation — lopping entries off the end leaves a shorter, self-consistent chain. Nothing inside the database can fix that. The fix has to be a witness its owner does not control.

The bundle records the log head it covered, and --seal-into-log records the bundle hash back into the log. Neither can be rolled back without contradicting the other:

clew evidence witness bundle/ --dsn "$CLEW_DSN"

$ `clew log` verify              # the log alone, after entries 2-3 were deleted
OK  1 entries, chain intact      # a short chain is a valid chain

$ clew evidence witness bundle/
FAIL witness   the log has no entry at seq 2, but this bundle recorded one.
               Entries have been removed from the end since this bundle was issued.

To make a truncation stick, someone would now have to collect every copy of every bundle ever issued. witness is a separate command from verify on purpose: verifying needs no credentials and must stay that way.

Signing is delegated, not invented

Clew seals — a SHA-256 manifest over every file, plus a bundle hash over the manifest. Standard library only, so anyone can check it.

Clew does not implement signing. A signature checkable only by someone holding the signing key is not a signature in the sense an assessor means, and inventing crypto here would be indefensible. Countersigning is detached and uses ssh-keygen -Y, which ships with OpenSSH and whose keys your organisation already manages:

clew evidence sign bundle/ --key ~/.ssh/id_ed25519
clew evidence verify bundle/ --allowed-signers allowed_signers

A signature from a key not in allowed_signers fails as "by someone this reader has no reason to trust". The seal is Clew's; who sealed it belongs to your key infrastructure.

The bundle is also a valid RO-Crate — adopted rather than invented, so it survives being handed to tooling that has never heard of Clew.

The CI gate

Everything above answers after the fact. The gate asks the opposite question, before anything runs: is any of this material something we are not allowed to use?

clew gate --pipeline sarek --samplesheet samplesheet.csv \
    --dsn "$CLEW_DSN" --gate-policy gate-policy.json --out clew-evidence/

CLEW GATE  clew/data/donors.csv
  blocking on     ConsentWithdrawn, QCFailed, SampleContaminated
  cleared by      ConsentReinstated, QCPassed
  log head        seq 5  0a9f436b2c7dca66

  BLOCKED  1
      donor_003    ConsentWithdrawn effective 2026-03-01, asserted by registry@example.org
          log seq 1, entry 01fd6803a761d58a
  UNKNOWN  2
      donor_002    the log holds no decisive fact about this subject
  CLEARED  2
      donor_001    QCPassed effective 2026-05-20, asserted by lab.qa@example.org

STOP  1 blocked, 2 unknown, 2 cleared

Exit 1 stops the build. Compliance as a build check, at the point where stopping is cheap.

Three outcomes, not two

BLOCKED, CLEARED, and UNKNOWN — the log has nothing to say about this subject at all. A gate that reports unknown as "not blocked" passes everything it failed to check, and goes green the day someone mistypes an identifier or points it at the wrong log. So unknown stops the build unless --allow-unknown says somebody decided otherwise, and even then it is still counted and reported as unknown rather than relabelled clean.

The report also says so outright when the log had never heard of any subject in the samplesheet — which almost always means the identifiers do not match between the two, not that everything is permitted.

Every way of failing to check exits non-zero

A green build must mean checked and permitted. If it can also mean could not check, the gate is decorative — and a decorative compliance gate is worse than none, because it manufactures a record of diligence that did not happen.

Decisions get reversed, and dates matter

For each subject the latest fact in effect decides, ordered by effective_from — when the decision was made in the world, not when it was entered. A subject withdrawn in March and reinstated in June is usable in July. Same-day facts are broken by log order, the only tiebreak nobody can back-date.

Which makes "was this run permitted when we ran it?" answerable:

clew gate ... --as-of 2026-05-01

as of 2026-05-01          as of today
  BLOCKED  3                BLOCKED  1
  CLEARED  0                CLEARED  2

Facts effective after the date asked about are ignored, so a historical gate result stays reproducible instead of changing every time someone records something new. This is what the log's two clocks were for.

It emits its own evidence

--out seals the result into a bundle, and clew evidence verify re-derives the gate decision from the bundled facts and the bundled gate policy — the same discipline as replaying a remediation plan:

  ok   files      6 files, all hashes match
  ok   log        5 entries re-chain to the recorded head (seq 5)
  ok   gate       passed=False; all 5 subject outcomes recompute identically

The shipped workflow at examples/clew-gate.yml uploads that bundle with if: always(), because the evidence of a refusal matters at least as much as the evidence of a pass — an artifact that only survives on green builds documents the days nothing was wrong.

The gate runs with read-only credentials. It asks questions; it has no reason to hold a role that can write facts, and CI is the last place to put one that can.

Clew ships gate-policy.example.json as a template, not a recommendation. Which of your event types should stop work is yours to author and yours to defend.

For auditors: a dashboard and a chat interface

Everything above produces evidence. These two read it, and both go through the same core layer — core/bundlestore.py to load bundles, core/query.py to answer — because two surfaces answering the same questions two different ways would eventually disagree, and on the day they did nobody could say which was wrong. A test asserts neither surface imports the other.

A page you can open from a USB stick

clew dashboard --bundles /path/to/bundles --out evidence.html

One self-contained HTML file. No server, no network, no scripts, prints legibly. It carries no timestamp, so regenerating it from unchanged bundles produces an identical file — two auditors comparing pages should be comparing evidence, not diffing dates.

What is not known sits above Findings, deliberately. A compliance dashboard that renders its gaps in small grey text below the fold is worse than no dashboard: it manufactures the impression of a clean bill of health out of an incomplete record. Withheld verdicts and subjects unknown to the log get counters of their own.

The page says outright that it is not the record — the bundles are — and every panel names the bundle hash it was drawn from.

An MCP server, so an auditor can just ask

clew mcp --bundles /path/to/bundles

MCP over stdin/stdout, stdlib JSON-RPC, no SDK. Point any MCP client at it:

Clew ships no model and calls none. It exposes tools; the auditor's own client supplies the conversation. That is the architecture, not modesty about scope — no AI in the decision path stays literally true, because every verdict was computed before the server started, by code that has never seen a prompt. A model cannot talk the tools into a different answer.

Read-only by construction. No tool writes anything and the server never opens a database connection at all — it reads sealed bundles. Recording a fact is clew log, run by a person with an actor identity, and an auditor's chat session is the last place a new fact should be able to enter a compliance record. There is a test asserting the server module contains no connection call, so a future convenience has to break it to land.

Every answer carries its citations, structurally — query.answer() refuses to return without them:

{ "kind": "log_entry", "seq": 1, "hash": "01fd6803a761d58a…",
  "actor": "registry@example.org", "effective_from": "2026-03-01" }

That matters because the consumer is a language model, and an auditor cannot tell fluent-and-wrong from fluent-and-right by reading it. Facts welded to their citations make a bad paraphrase checkable rather than merely persuasive. The guardrail the server hands the model at startup says it plainly: quote the citations, read the coverage out loud, and never conclude compliance — there is no tool here that says an obligation was met, and the model must not supply one.

That is a guardrail, not a guarantee. Assume the prose is a convenience and the citations are the record.

A log has no identity, and that is handled rather than hidden

Two logs both number their entries from 1. Merging bundles sealed from different logs would interleave two unrelated histories into one plausible timeline, and no field distinguishes them. So the loader compares what must agree if the logs are the same — an entry at a given sequence number has one hash — and says so loudly when they do not:

THESE BUNDLES DISAGREE: 2 sequence numbers (including 1, 2) carry different
entries in different bundles, which means they were sealed from DIFFERENT
LOGS. Answers drawn from the combined history are not trustworthy.

The warning travels on every answer drawn from the merged entries, and the dashboard renders it as a stop panel above everything else. Giving the log a real identity is the proper fix and is not built.

What Clew claims, and what it does not

Clew is a system of record, not an attester. It claims three things, all checkable: the computation is deterministic, the result follows from the inputs, and anyone can re-run it and get the same answer. It claims nothing about whether your inputs were true or your policy was right. Your tags are inputs, not Clew's claims.

Honest edges, reported rather than hidden:

• Traversal follows derivation and stops at influence. A published finding that informed a business decision is recorded as a terminal reference, not pretend-tracked.
• Uninstrumented systems are reported as unknown, never as clean.
• Clew proves the record was tombstoned and nothing later referenced it. It does not prove physical destruction. No cryptography reaches a freezer.

Architecture

Two streams meet in the middle. The left one is machine facts — whatever the engine recorded, merged into one graph. The right one is human facts — things no engine can know, translated by the domain layer into graph terms. The core knows nothing about either world: it traverses, applies the versioned policy, and logs.

flowchart TB
    subgraph NF["Nextflow already writes this"]
        LS["Lineage store (25.04+)"]
        RC["nf-prov RO-Crate"]
        WS["work/ symlinks (any version)"]
    end
    subgraph PA["People assert this"]
        W["Withdrawal"]
        P["Publication"]
        D["Tool defect"]
    end
    subgraph CORE["core/ — the engine, zero domain vocabulary"]
        T["Traversal<br/>blast radius, classes"]
        POL["Policy<br/>versioned, hashed"]
        LOG["Log + evidence<br/>append-only, replayable"]
    end
    NF -- extractors --> G["One stitched graph<br/>every run, one JSON"]
    PA --> DOM["domains/<br/>donor &rarr; graph nodes"]
    G --> CORE
    DOM --> CORE
    CORE --> PLAN["Remediation plan<br/>delete, re-run, disclose"]
    CORE --> GATE["CI gate<br/>blocks bad inputs"]
    CORE --> AUD["Dashboard + MCP<br/>answers with citations"]

clew/core/     traversal, contribution vocabulary, versioned policy, event log,
            evidence bundles, the gate, the query surface. Zero domain
            vocabulary.
clew/domains/  the layer allowed to know about sarek, samplesheets, donors.
tests/         288 tests, stdlib unittest.

The boundary is enforced by a grep: clew/core/ must never mention a sample, a donor, a consent, or a workflow engine, and must import nothing from domains/. Adding a new domain (say, AI training data with opt-out semantics) means adding a directory, not editing core. That grep is tests/test_core_boundary.py — an unenforced rule stays true right up until it doesn't.

python3 -m unittest discover -s tests

263 of the tests need nothing installed. The other 25 exercise the log's storage behaviour — the role grants, the triggers, concurrent appends — and skip unless you point them at a database you own:

CLEW_TEST_DSN=postgresql://user:pw@localhost:5432/clew python3 -m unittest discover -s tests

The log's arithmetic is deliberately on the other side of that line. Hashing and chain verification are pure functions on plain dicts, so an auditor checking an exported bundle needs a JSON file and an interpreter — not a database driver and a server. "Anyone can check this without us" has to be a fact rather than a slogan.

Status

Working: lineage extraction from real runs, blast radius for three trigger types, contribution classes with fail-closed defaults, remediation plans, publication assertions, mixed verdicts from a single traversal, the append-only event log, the versioned remediation policy, sealed evidence bundles that replay offline, the CI gate, and the auditor surfaces — an offline dashboard and a read-only MCP server.

Not built: a log identity, so bundles from different logs are detected rather than distinguished. A donor-facing transparency log. Any domain beyond nf-core pipelines.

Contributing

Issues and pull requests are welcome — especially from people who run pipelines for a living and can say where the model is wrong. See CONTRIBUTING.md; a first pull request needs the CLA agreement described there, an issue needs nothing at all.

License

AGPL-3.0.