manuscript-guard

Make every number in a scientific manuscript traceable to its source.

A number in a paper comes from one of three places: your results, the literature, or a convention of scientific writing. manuscript-guard holds a manuscript to that. Numbers from your analysis are bindings into a results file that your code writes, numbers from the literature are bindings into a ledger backed by stored sources, and anything else has to be justified. Change the analysis, run it again and rebuild, and every bound number in the manuscript and the supplement follows. A number with no source is a finding of the check, not something left for a reviewer to notice.

It is a command line tool written in Python, for a manuscript written in Markdown and built to a Word document. The checks are deterministic code: they run offline and in CI and give the same answer every time. No gate asks a model anything. Two of them read a recorded review, of each figure and of the manuscript, which a person or a model may have written: the record is what the gate reads, and a person answers its major findings (principles). Skills and hooks for Claude Code, Codex and other agent tools are optional, and help with the parts that need judgement.

What it catches

Each row names the stage from which its finding fails the run. Before that stage the finding is printed and counted, and the run passes: a new project starts at design, where the manuscript can wait (stages).

check knows that a results file is out of date from digests: of the script that wrote it, and of the inputs that script declared. It does not follow imports, so a change in a helper module fails the run only where the script lists that module among its inputs. Left unlisted, a source file under analysis/ that is newer than the results gets a warning, and the run passes. manuscript-guard verify asks the other question. It runs the analysis again into a scratch copy and compares every value, which takes as long as the analysis does, and is what catches a result that changed while its file did not.

This is the finding for the first of those, on the worked example, after one binding in the abstract was replaced by the number it stood for. The line that opens the output and what follows the finding are left out:

And for the second, after one value in a results file was changed by hand. ... stands for a line left out, and other findings follow from the same edit:

  [FAIL] G1 results/01_disproportionality.json
         01_disproportionality.json has been modified since the analysis wrote it
         ...
         hint: results are machine-written; change the analysis and re-run it rather than editing this file

How it works

The manuscript source is Markdown. Numbers appear in it as bindings:

The database contained {{results.cohort.n_reports}} reports, of which
{{results.cohort.n_drug_reports}} named example-drug. The reporting odds ratio was
{{results.ror.point}} (95% CI {{results.ror.ci_low}} to {{results.ror.ci_high}}).

Your analysis publishes those values through an emitter, which records where they came from. There is one for Python and one for R, and anything that can write JSON can take part:

from manuscript_guard.emit import Emitter

em = Emitter(__file__, inputs=["data/reports.csv"])
em.value("cohort.n_reports", 4000)
em.value("ror.point", 3.4211, digits=2)
em.write()

Then two commands do most of the work:

manuscript-guard check     # every gate, against the source
manuscript-guard build     # the .docx, with live Zotero citations

check reads the source, where a binding is still visible, and not the rendered text, where every number looks alike. The rule is short: in manuscript source, a bare number is a defect unless it is a recognised convention (a 95% confidence interval) or a pointer (Table 1). The check also runs the other way: a value the analysis declares as quoted, which nothing quotes, fails too.

build writes the .docx from the Markdown every time, so the document is never edited and nothing is carried across by hand. With Zotero open it writes live Zotero citation fields that Word's plugin adopts. Without it, --offline formats the citations from a committed references.bib, which is what CI and a co-author without your library get.

What is checked

In place of a gate, a finding can carry one of two labels that are no gate of their own: G0, for a file of the project that cannot be used as it stands, and BUILD, for what the build would refuse or warn of.

You do not have to satisfy every gate on the first day. Each finding declares the stage at which it starts to matter, from design to submission, and until then it is printed and counted without failing the run: see stages. What each gate guarantees and why it is built that way is in principles.

The part no gate can check

Every gate above asks whether a number is the number the analysis produced, or whether a quote is really in the stored source. None can ask what a reader of that source asks: does the sentence say what the document says. A value can be right, its quote can be right, and the sentence around it can still name the wrong denominator, subgroup or year with every check passing. On the first manuscript written with this toolkit, that is what happened, and a co-author reading one item against one quote is what found it.

manuscript-guard checker build --for "A Co-Author"      # one file to send them
manuscript-guard checker import --answers answers.json  # what they said, into checks/decisions.csv
manuscript-guard checker status                         # outstanding, stale, and who disagrees

build writes one file. It carries every item, the sentences each value appears in, and the evidence rendered into the file itself: images as data URIs, tables and excerpts as data. It opens by double-clicking, in any current browser, offline — nothing installed, no account, no network, and nothing sent anywhere by the page. The person presses a button and sends back the answers file.

What is asked is found from the project's own files: the values quoted from the literature with the passage each came from, the sentences that cite something, the references, and the authors. A number transcribed from a table in a source document is not among them and cannot be — only the project knows which row it came from — so a project that traces its own numbers adds them in checks/items.json.

A group the project fills, it fills alone: one contributed item in a group means none of the toolkit's are added to it, since the two are keyed differently and a co-author would otherwise read the same value twice.

Every answer carries the digest of the item as its maker saw it. If the sentence or the value moved after they looked, the answer is not recorded as agreement: their yes was about text nobody has now, and the item is outstanding again. checks/decisions.csv is append-only, because "was this checked, by whom, and when" is a question asked months later, by whoever is answering a reviewer.

What this gives you is a record of who looked, at what exactly, when, and what they said. It is not a proof that they read it correctly, and nothing here checks their reading.

Install

Python 3.10 or later:

pip install manuscript-guard
manuscript-guard --version

To build a .docx you also need pandoc. Zotero, R, and pdftotext or pypdf for reading PDFs, are needed only for particular things, and the tool says which when you reach it. Installing and upgrading has the table of optional tools, the upgrade commands, how to install from the repository, and the R emitter.

Quick start

Start a paper:

manuscript-guard init my-paper

That writes a project with a paper.yaml, an analysis plan to fill in, a manuscript and an AGENTS.md that gives any agent working there the rules on one page.

Or run the worked example first. example/ is a synthetic pharmacovigilance study that exercises every gate but G13, having no revision round. It is in the repository and not in the installed package, so it needs a clone, a Python that has manuscript_guard installed, and matplotlib for its figure:

git clone https://github.com/BasileChretien/manuscript-guard
cp -r manuscript-guard/example my-example && cd my-example
python analysis/00_simulate.py && python analysis/01_disproportionality.py && python figures/forest.py
manuscript-guard check
manuscript-guard build --offline     # writes build/manuscript.docx and build/supplementary.docx

Then type a number over one of the bindings in manuscript/main.md and run check again. A different matplotlib version draws a slightly different figure, and check then lists the figure's recorded review as stale; that is INFO until the internal-review stage.

When a finding surprises you, manuscript-guard explain manuscript/main.md prints every number in the file and the rule that classified it.

Commands

Each takes --help. verify is a separate command because it executes your analysis, which a gate never does, and because it takes as long as the analysis does.

Agent tools

The pip package is the whole guarantee and needs nothing else. An agent tool adds two things: skills, for the parts that need judgement, and hooks, which catch a mistake at the moment it is made. Agent tools says what each tool enforces and what it does not.

The Claude Code plugin

This repository is its own plugin marketplace:

claude plugin marketplace add BasileChretien/manuscript-guard
claude plugin install manuscript-guard@manuscript-guard

Fifteen skills. Start with project-setup; most of the others name the finding codes that should send you to them.

Four hooks: one line on the project's state when a session starts, a refusal to edit files that are machine-written, a note on the numbers just saved in a manuscript file, and the submission check before a command that looks like a submission.

Codex

Codex installs the same plugin, with the same skills and hooks:

codex plugin marketplace add BasileChretien/manuscript-guard
codex plugin add manuscript-guard@manuscript-guard

By Codex's documentation, it runs a hook only after you have trusted it, under /hooks.

Gemini CLI, Mistral Vibe, Kimi Code CLI and other tools

By their own documentation these read skills from a folder, and one command copies the fifteen there:

manuscript-guard install-skills              # for you, in every project: ~/.agents/skills
manuscript-guard install-skills --project    # for one paper: .agents/skills in that project

There are no hooks under these tools. check, build and submit hold as they do everywhere.

More

Status

Alpha. Its author has written one real manuscript with it, and the gates still change from week to week: interfaces, file formats and what a gate reports can change without a migration path. Every round of review so far has found something the round before had opened up.

A passing check means that no finding due at the project's stage is open. At submission that is what the tables above describe; at an earlier stage the findings not yet due are printed and counted, and the run still passes. Neither is evidence that a paper is right. What the toolkit cannot see is listed under "Known gaps" in DESIGN.md, because a gate whose limits are undocumented gets trusted beyond them.

The most useful contribution now is a case where a gate is wrong, in either direction: CONTRIBUTING.md says how.

Citing

If you use manuscript-guard for a paper, the "Cite this repository" button on GitHub gives the reference, from CITATION.cff.

Licence

MIT.