papersmith-ai

A paper forge: turns reference PDFs into high-fidelity Markdown that an agent can read (equations in LaTeX, tables as tables, figures as files), and drives the formal deliberation —mathematics and experimental design— through to the code that tests itself and the compiled paper.

This README is the way in: what papersmith is, how to install it, the order of the work, the workspace from the inside, development and troubleshooting. The reference pages live under docs/: every skill in docs/skills.md and every smith command in docs/cli.md.

Contents

1. If this is your first time
2. Use flow — the short path
3. Getting started
4. How it works — the order
5. Inside the workspace
6. Harnesses and skill projection
7. The agents
8. Anatomy of each skill
9. The CLI commands
10. Remote compute, carefully
11. Architecture
12. Development and testing
13. Troubleshooting
14. Related documentation
15. License

If this is your first time

Welcome. This forge is built to take a PDF to a compiled paper without guessing the order. The minimum path:

1. Install the CLI and set up the environment — see Getting started.
2. Create your workspace — smith init ~/papers/my-paper --title "…" --topic "…". What it leaves inside, in Inside the workspace.
3. Check that the idea is plausible — with your idea in two sentences, /plausibility tracks the SOTA and returns the constellation plus a ranked top 5. If you have no clear sentences yet, the dashboard's phase-2 box offers an optional Ayuda temática: type 3 to 5 keywords and it proposes the idea for you to edit.
4. Put a reference PDF in guidance/reference-papers/ and run smith ingest <pdf> — or ask for /paper-ingestion inside your harness.
5. Open the deliberation with /proposal-deliberation and discuss the mathematics with the agent.

Not every skill has to be used, or in this order: each one declares what it needs first and refuses before doing something half-done. If an input is missing it doesn't invent it: it exits with a rejection code that names what's missing. A PAPER_ABSENT reports a missing input, not an operator error.

The full map is in Use flow; the detailed route, in How it works — the order. If an AI agent is used as a copilot, the ones already defined are in The agents.

Use flow — the short path

Every skill is invoked by name in Claude Code (/paper-ingestion) or requested in Spanish ("ingiere los papers"). Not all of them have to be used, or in this order: each one declares what it needs first and refuses if it's missing, so an invocation from the middle of the flow reports which input is missing.

The full path, from a PDF to a compiled paper:

The minimum to try the forge is step 1: with the idea in two sentences, /plausibility tracks the SOTA and returns the constellation plus the top-5. Only afterwards, if desired, a PDF can be placed in guidance/reference-papers/ and run through /paper-ingestion; the deliberation is then carried out with /proposal-deliberation. Steps 5 to 7 only make sense once there's a published proposal and a target repository to implement it in.

The diagram shows the full flow: every stage in this order with its command, its agents (with the tools each one declares in Pi), its human gates and what it leaves on disk. The interactive version, with links to the source of each node, is in docs/diagrams/papersmith-pi-flow.html (opens in the browser; generated with Archify, and its source and its recipe are versioned in docs/diagrams/).

The dashboard (smith ui) opens on this flow and shows the same order as eleven phases under Pipeline y Avance. Below the diagram, the Flujo Archify tab lists every agent the workspace ships (/api/agents): its role, the skill it loads, its stretch, its tools, the phase(s) it acts in and the harnesses it is projected to, in a table you can sort and filter. Clicking a phase opens its console: a text box with the stage's rules as a guide that sends /<skill> <text> to an agent in the support terminal (for /plausibility, the idea in at least two sentences, or the optional Ayuda temática described under Step 1), the real smith command for the phase, or a link to the panel that owns the work: the concept map, the math tutor, the PoC board, the sections, the PDF preview or the gates. After "Ejecutar", the drawer tracks the run step by step until the server finds the phase's evidence on disk and marks it verified; the next phase then unlocks and lights up. Each phase is locked until the one before it passes (override available). A running phase also shows in a dock at the bottom left on every tab. A click opens its tracking panel in place, with the checkpoints, the agent's terminal and ⏹ Detener proceso, which interrupts the agent's turn and keeps its session. Closing the CLI, changing tabs or reloading the page never ends the session of a running phase: a reload reattaches to it. The same eleven phases are written into every harness entrypoint (CLAUDE.md, OPENCODE.md, .pi/APPEND_SYSTEM.md, .agents/AGENTS.md) under "Paper construction flow".

The dashboard has twelve tabs; the tab names below are the Spanish ones the dashboard shows (the interface also has an English catalogue):

The novelty assessment and its gate. Novedad & Hallazgos (new in 0.45.0) sits between the experiments and the manuscript and has four parts: a matrix of hypotheses against the empirical evidence of both stages (metric, criterion, observed result, test, rationale and verdict badge, read from each product's Results/discussion.json); the novelty claims contrasted with the sota-pool/candidates.json candidates and the baselines; a checklist for defending the work before reviewers (leakage-free partitioning, significance tests, empirical support, SOTA contrast), derived from what the discussions record; and the decision gate. G-NOVELTY ("Novelty & Findings Gate") turns ACTIVE once Phase 5 passes, and PASS when you approve. You decide from the panel's buttons or from the terminal with smith novelty {status,approve,iterate,reset}: approve records that the evidence supports the novelty, and iterate records that the work returns to experiments, proposal or plausibility. The decision and its history go to .papersmith/novelty_gate.json. The gate records your decision; it does not lock a later phase on its own.

Phase 5 gating. The sub-phases that close the experiments read the results discussion, not just a folder. 5.8 (proof of concept) and 5.11 (full experiments) pass only when a product of that stage holds a complete Results/discussion.json — a bare Results/ folder reads as active — and Phase 5's progress follows them; 5.12 (code review) reads Results/reviewer_report.json. Scaffold and check the discussion with python skills/_core/results/discussion_cli.py scaffold|check|status --product implementations/<repo>/<Name>. scaffold copies each experiment's hypotheses and success criteria verbatim and leaves every verdict pending; check rejects an evidence path that does not exist and, at the full stage, a decided hypothesis that cites no significance test. The schema is in skills/_core/results/README.md.

Choosing and adapting to a journal

Where a paper goes is a decision of its own, and it gets evidence rather than a hunch. From the Previsualización PDF tab you choose a quartile range and press Buscar revistas (or press Analizar con agente, which types /journal-finder <range> into the CLI assistant). The CLI turns the paper's title, abstract, keywords and references into a ranked shortlist of journals that already publish work like it. Nothing is ever submitted.

• How the fit is scored. fit_score = 0.45 * similar_works + 0.3 * cited_in_paper + 0.25 * topic_overlap, each component divided by its maximum over the journals evaluated: works like the paper that the journal published (one OpenAlex query of quoted key phrases, last six years), references of paper/refs.bib published there, and the overlap between the paper's OpenAlex topics and the journal's (with the fallback chain, the paper's keywords and facet phrases against the journal's SJR categories and DOAJ subjects). A component that is zero for every journal drops out and the other weights are rescaled. When the title applies a method to a domain (the tail after for, applied to, toward), the search runs one query per facet, method and application, and a journal that shows no application evidence has its score halved and carries the no-application-evidence flag; it still appears, flagged. The numbers are in the JSON (fit, facets, query.scoring) and in the panel, which prints the formula above the table.
• Where the journals come from. --source auto|openalex|crossref picks the chain (default auto; the panel's source selector sends the same choice). OpenAlex answers first. When it cannot (no key and the keyless demo allowance is spent, a 429, a 401/403, a 5xx or a network error) auto does not refuse: it continues with free sources and says so in the output (fallback: {used, code, reason}, a note and, in the panel, a banner). The fallback chain is Crossref works search for works like the paper (the polite pool: set PAPERSMITH_CONTACT_EMAIL and it travels as mailto), Europe PMC for biomedical papers (a keyword or an application phrase such as EEG, clinical or brain triggers it), DOAJ for open-access state, APC and review process, and Semantic Scholar only when SEMANTIC_SCHOLAR_API_KEY is set. Works are aggregated per journal by ISSN, else by title. --source openalex keeps the old behaviour and refuses when OpenAlex is unavailable; --source crossref skips OpenAlex. The JSON names who answered (providers, and sources on every candidate), so a fallback result is never passed off as an OpenAlex one. The fallback has less to go on: no OpenAlex topics, no citedness and no proxy quartile.
• Where the quartile comes from. The SJR quartile comes from a SCImago CSV that you download once from https://www.scimagojr.com/journalrank.php ("Download data") and import (python3 skills/journal-finder/scripts/journal_cli.py sjr-import --file <csv>, or the panel's upload, which sends the file as the raw body of POST /api/journals/sjr, up to 16 MiB); scimagojr.com answers automated downloads with a browser challenge, so the forge never scrapes it. When OpenAlex answered, a journal without an SJR row gets an OpenAlex proxy (the share of journals of the same field with a higher 2-year mean citedness), always labelled openalex-proxy, in the JSON, in the panel and in the agent's markdown: it approximates a citation percentile and is not an SJR quartile. No free source besides SJR and that proxy gives a quartile, so without OpenAlex the quartile is SJR or it is unknown (quartile: null, quartile_source: "unknown"). Journals of unknown quartile are excluded from a quartile range by default and counted in excluded_reasons.quartile_unknown; --include-unknown-quartile (the panel's toggle) keeps them, flagged quartile-unknown. Import the SJR file or get an OpenAlex key and the question goes away.
• The review-time estimate. Per journal, the median and interquartile range of the days between received and accepted over its latest 40 research articles of the last three years, read from Crossref first, then PubMed, else none (fewer than five dated articles reads as none). It is an estimate of submission-to-acceptance, revision rounds included; it is not the time to first decision.
• The OpenAlex key, its budget and the response cache. OpenAlex now requires a free API key for real use; keyless calls are a small demo allowance (USD 0.10 per day, shared by everyone behind one IP). A free key takes an account at https://openalex.org/signup with a verified email, then the key from https://openalex.org/settings/api, and export OPENALEX_API_KEY=... (the CLI sends it only when set, and never asks for it on your behalf). It gives USD 1 per day of usage, ten times the keyless budget. By OpenAlex's price list (checked 2026-10-31), fetching a record by id is free, a list with a filter costs USD 0.10 per 1,000 requests and a search USD 1 per 1,000, so a day's budget is about 1,000 searches or 10,000 filtered lists. The rate limit is 30 requests per second with a key (10 without) and 5 per second for searches with more than 5 boolean operators; the budget resets at midnight UTC, a refused request costs nothing, and the data is CC0. The finder's own cost is an estimate from its call pattern, not a measurement: a search makes two or so search-priced calls for similar works, up to 15 more when references name journals without a DOI, up to 4 DOI-batch lists, and, for each journal without an SJR row, two filtered lists for the proxy quartile (up to about 130). That is roughly USD 0.01 to 0.04 per search, fewer with an imported SJR file; api_calls.budget reports what OpenAlex says is left. The CLI keeps successful answers (of every service) in ~/.cache/papersmith/journal-finder/http/ (PAPERSMITH_JOURNAL_CACHE_DIR moves it): a search is reused for one day, any other lookup for seven, and an error is never stored. The panel's forced-refresh option, or --no-cache, skips the cache for one search. The result's api_calls reports the calls made and served from the cache per service and the budget state, and the panel shows it. When OpenAlex answers 429 asking for a wait of more than a minute the quota is spent and the CLI does not retry: with auto the search continues with the fallback chain and a note says how to get the key; with --source openalex the refusal is NETWORK_UNAVAILABLE and its message names OPENALEX_API_KEY; later in a search that step degrades with a note and the budget reads exhausted.
• From a journal to its template. Elegir esta revista runs venue select and opens paper/venues/<slug>/. The state machine is selected → requirements → template → audited → planned → approved → built (→ compiled); planned and approved are owed only while the audit holds a blocker finding. The venue-adapter agent reads the author guidelines (every requirement it records carries the quote and the url it came from), venue diff measures the manuscript against them, and the agent proposes an adjustment plan only when one is prudent.
• The operator approval gate. The plan is approved by you, with your own words, bound to the plan's sha256: venue approve --statement "..." --plan-sha256 <hex> or the panel's Aprobar plan button. The button's route, POST /api/venues/<slug>/approve, refuses a request with no Origin header (curl, an agent's shell) as well as a foreign one, so an approval recorded via: dashboard always came from the page; from the terminal you give your words to venue approve. A revised plan voids the approval, and venue build refuses while a blocker finding has no approved plan. The approved text changes are applied one block at a time through the existing write loop, never by the agent that proposed them.
• The canonical manuscript is never overwritten. Everything for one journal lives under paper/venues/<slug>/ (venue.json, requirements.json, audit.json, plan.md, approval.json, main.tex). paper/main.tex and sections/ are never written by any venue verb, and the journal's main.tex carries the canonical blocks byte for byte. The panel compiles it in its own directory and previews its PDF beside the canonical one.

The skill's contract is skills/journal-finder/SKILL.md, the venue verbs' is the "Venue adaptation" section of skills/paper-writing/SKILL.md, and the guided tour is in docs/skills.md.

1. Nothing produced gets uploaded. proposals/, experiments/, paper/, implementations/ and sota-pool/ travel to GitHub as empty folders and nothing more; guidance/ versions its folders and never the PDFs or the Markdown inside. The scaffolding gets versioned, the content doesn't. A half-written paper is research material, and publishing it by accident is a leak.
2. Each skill refuses before doing something half-done. If an input is missing, it doesn't invent it or skip it: it exits with a rejection code that names what's missing. A PAPER_ABSENT or a SECTION_CONTRACTS_UNREADABLE names what the skill is missing, not an operator error.
3. The operator closes the deliberation. The two deliberation skills propose, discuss and prepare the edit, but neither publishes without the operator's acceptance. It is part of each one's contract, and the engine enforces it.

Getting started

None of this requires keys or external services: the forge runs locally.

0. What your machine needs to have

Two runtimes, and both are mandatory: the forge uses Python for the CLI and Node for the deliberation engine, and without the second one half the framework won't start.

Both are checked at once:

python3 --version && node --version && npm --version

If Node is missing, smith init doesn't fail: it warns with npm was not found; deliberate requires npm install for jiti/typebox and moves on, because the rest of the workspace can be created. But proposal-deliberation and experimental-deliberation —the two skills that propose and discuss before writing— don't run without those dependencies. To create the workspace without them, init --no-npm does so explicitly.

The ingestion runtime from step 2 is separate and is optional: it's only needed to extract PDFs with OCR.

1. Install the CLI and set up the environment

# 1a. The CLI, from the latest version published on PyPI
pipx install papersmith-ai
#     the same with pip, for setups without pipx
pip install papersmith-ai
#     or the tip of `main`, without waiting for a release
pipx install git+https://github.com/LIAUNAL/papersmith-ai.git

# 1b. Or from a clone of `main` (for framework development)
git clone https://github.com/LIAUNAL/papersmith-ai.git && cd papersmith-ai
pipx install .            # with pip: pip install .

# 1c. Update the CLI when a new release comes out
pipx upgrade papersmith-ai
#     `upgrade` doesn't see a git-URL install: reinstall
pipx install --force git+https://github.com/LIAUNAL/papersmith-ai.git
#     with pip, `--upgrade` over a git URL doesn't always re-clone
pip install --upgrade --force-reinstall "papersmith-ai @ git+https://github.com/LIAUNAL/papersmith-ai.git"

# 1d. Bring existing workspaces up to that version
smith upgrade --plan-migrations ~/papers/sparse-ae  # which artifacts are missing, without writing anything
smith upgrade ~/papers/sparse-ae                    # syncs the files and migrates the artifacts
smith status ~/papers/sparse-ae                     # drift and pending migrations

# 2. Isolated ingestion runtime (micromamba: Python 3.12, PyTorch, Surya OCR, llama-server)
python3 scripts/setup_env.py install

# 3. Node dependencies and projection of skills to the harnesses
npm install
npm run setup:harnesses

The command is smith. Installing papersmith-ai puts smith on PATH, and every command in this README uses it (smith init, smith upgrade, smith status, …). The older papersmith entry point is still installed as a compatibility alias for existing scripts; it runs the same CLI and reports itself as smith.

The package is published on PyPI. pipx install papersmith-ai brings the latest release, which pipx upgrade papersmith-ai in 1c updates. The git URL in 1a brings the tip of main, for framework development. The neighbouring name papersmith —without the -ai— is also on PyPI, but it's another project (Word document generation), so pip install papersmith doesn't install this forge.

main is the only branch that's maintained, and each release also gets its tag, so an exact build is frozen by pinning it in the URL — for example pipx install --force "git+https://github.com/LIAUNAL/papersmith-ai@v0.57.0". 1b takes the clone as it is, including uncommitted changes; a reproducible build comes from 1a or a tag. The changelog is in CHANGELOG.md.

Updating is two steps and they're two different objects. 1c reinstalls the CLI from main; smith --version confirms the installed version. 1d moves each workspace that already exists, and it isn't optional: a release can change the shape of an artifact a workspace has already produced —the graph pool's atlas.html, for example—, so syncing the framework files without migrating the artifacts leaves the new scaffolding reading old state.

smith upgrade syncs the workspace with the installed version and applies that version's artifact migrations. The version the workspace records doesn't advance until those migrations are done, so smith status may report that it hasn't yet reached the release just installed. --plan-migrations reports what's pending and finishes without writing anything; it isn't called --dry-run, because it doesn't preview the kit copy. --no-migrate syncs the files and leaves the artifacts as they are, with the version marker where it was.

Which workspaces can be upgraded. Re-initializing a live workspace is never the answer: smith init in a fresh directory leaves behind every ledger, receipt, seal, managed revision and lock the paper has accumulated. Workspaces recorded at 0.12.0 or later are supported, and since 0.14.0 every release that changes the shape of something a workspace already has ships its migration in the same release (or a compatible read that keeps reading the old shape). A workspace older than 0.12.0 also upgrades —the files are synced and the convergence migrations run—, but shape changes from before 0.12.0 shipped no migration and leftovers may remain (the .antigravity/ folders of 0.8.0 are not removed). With no recorded version, the version-gated migrations stay undetermined and are not applied; a version that cannot be ordered, or one newer than the kit, is refused unless you pass --allow-downgrade. The maintainer-side policy is in docs/releasing.md.

The two upgrades are separate: neither one touches the other. smith upgrade refuses to take a workspace to a version older than the one it records, unless --allow-downgrade is passed.

scripts/setup_env.py install provisions the whole ingestion stack in a separate micromamba environment (CPU or CUDA, depending on what it detects), including the llama-server binary that the OCR needs — it is not Ollama nor a Homebrew installation. The micromamba it bootstraps is a pinned release whose SHA-256 is checked before it runs; the comment above MICROMAMBA_VERSION in scripts/setup_env.py is the procedure for moving the pin. npm run setup:harnesses symlinks the canonical skills/ tree into .claude/skills, .pi/skills, .opencode/skills and .agents/skills, with relative, idempotent symlinks (see Harnesses and skill projection). The first run of smith ingest downloads ~1.5 GB of Surya weights if the environment wasn't pre-provisioned.

2. Create a workspace

smith init ~/papers/sparse-ae \
  --title "Sparse Autoencoder Audit" \
  --topic "mechanistic interpretability" \
  --remote kaggle

cd ~/papers/sparse-ae
smith status
smith status --json          # machine-readable snapshot

The workspace stays decoupled from the framework checkout: the scaffolding is synced with smith upgrade, and the research artifacts are protected by contract. What init creates exactly —and what gets versioned— in Inside the workspace.

3. Ingest literature

smith ingest https://arxiv.org/abs/2309.08600    # arXiv, OpenReview or URL
smith ingest ~/Downloads/reference-paper.pdf     # local PDF
smith ingest ~/Downloads/scan.pdf --ocr       # scanned: balanced OCR mode

Each loose PDF ends up in its own folder, next to its Markdown (text + LaTeX + tables, with the bibliography stripped) and the images of its figures. The pipeline's details, in the paper-ingestion section of the Anatomy.

4. Deliberate, implement and run

smith deliberate . --action status                                   # state of the managed proposal
smith implement . --action verify --target implementations/my-repo   # over an already materialized repo
smith run smoke_and_invariants --dry-run                             # plans; dispatches nothing
smith novelty status                                                 # hypotheses vs evidence and the G-NOVELTY gate
python skills/_core/results/discussion_cli.py check --product implementations/my-repo/MyPaper   # a stage's results discussion

smith novelty also takes approve, iterate --stage {experiments,proposal,plausibility} and reset; the discussion tool takes scaffold, check and status. Both are described in The dashboard above and in docs/cli.md.

Inside a harness, the same capabilities are requested by skill name: /paper-ingestion, /proposal-deliberation, /experimental-deliberation, /proposal-implementation, /experimental-implementation, /paper-writing. The ones that don't depend on a domain — /kaggle-accounts, /remote-execution, /skill-audit — act as gatekeeper, memory and auditor of what is sent out.

How it works — the order

Step 1 — Check that the idea is plausible (plausibility)

Before there is anything to ingest there is an idea, and this is the step that tests it. It needs no PDFs, which is why it comes first: all three stretches work from the idea and from what can be found about it.

Everything lands in sota-pool/, which is ignored like guidance/. Nothing here files into guidance/: this step reads the literature, it does not collect it. The pool it leaves behind is what proposal-deliberation shows alongside the mathematics, and its ranked top 5 is what tells the ingestion step which PDFs are worth having.

An idea stated in fewer than two sentences is refused before the search starts: scouting an unstated idea invents its own question. A claim with no abstract passage behind it does not become a planet — a quoteless planet is a drawing.

Thematic help, when the idea is not clear yet. In the dashboard, the box that opens when you click phase 2 carries an optional Ayuda temática. Type 3 to 5 keywords and the server asks the lab's banco-lia MCP (its search_bank tool) for the best matches; the panel lists the ranked LIA items (projects and articles), the topics they fall under and a proposed /plausibility idea, composed from your keywords and those matches, in an editable text area. Nothing runs until you press Aprobar y lanzar, which sends /plausibility <your edited idea> through the same launch path as any other phase-2 prompt, so the two-sentence rule still applies. The bank items are LIA continuity context, never SOTA evidence. It needs uv/uvx on PATH and the banco-lia entry of the workspace's .mcp.json; the first run downloads the package and later runs work offline. If the bank cannot be reached the panel says why and you write the idea yourself. Details and the endpoint (POST /api/plausibility/suggest) are in docs/mcp.md.

Step 2 — Put each PDF in the folder according to its role

All three folders travel empty. Each one arrives in a clone with its .gitkeep and nothing more: guidance/*/* is ignored, with !guidance/*/.gitkeep as the only escape hatch. The scaffolding is versioned, the PDFs and their Markdown are not. tests/test_forge_scaffolding.py derives both halves —what the profile.ts declare as source against what git actually tracks— and goes red if a declared folder stops travelling.

Step 3 — Ingest the PDFs (PDF → Markdown)

In Claude Code, the skill is invoked:

/paper-ingestion

(or, in Spanish: "ingiere los papers"). For each loose PDF, it creates a folder with the paper's name, moves the PDF inside and writes a lightweight <name>.md (text + LaTeX + tables, with the bibliography stripped) together with the images of the figures:

guidance/reference-papers/computers-13-00176-v2-1/
├── computers-13-00176-v2-1.pdf
├── computers-13-00176-v2-1.md
└── _page_4_Figure_2.jpeg   (figures that the .md references)

A loose PDF (directly in a source root folder) is pending; a paper that's already inside its folder is skipped. Re-ingesting means deleting the paper's folder (leaving the PDF loose) and running the skill again. The configuration lives in papersmith.yaml (source_roots, mode, strip_references, force_ocr). If a PDF's tables come out scrambled because its text layer is bad, --force-ocr (or force_ocr: true) rebuilds every page by OCR instead of reading that layer.

Step 4 — Deliberate over a proposal (proposal-deliberation)

In Claude Code, the skill is invoked:

/proposal-deliberation

On the first turn it automatically loads the Markdown from guidance/paper-guide/ as context and acts as a mathematics tutor. The available actions:

• describe an idea and ask for a first version,
• ask for edits to a managed proposal,
• run the lifecycle of managed revisions.

Proposals live in proposals/, one per managed revision (research-concept-rNN.md).

Each published revision travels with its own constellation: the SOTA's with the proposal inside, in proposals/research-concept-rNN.graph.html, drawn by the same 3D viewer as the atlas and named from the revision. The deliberation's publication stretch produces it, from the proposals/research-concept-rNN.sky.json overlay written during the discussion: there the main idea, the topics the proposal takes from the SOTA papers, its novelty, the problem it would solve and three to five concepts remain as nodes and edges instead of as sentences. If that overlay doesn't exist, the stretch reports it as pending instead of drawing relationships nobody agreed on.

Step 5 — Take the proposal to code (proposal-implementation)

In Claude Code, the skill is invoked:

/proposal-implementation

It takes the current revision and materializes it in a target repository, which lives in implementations/ with its own git and its own virtual environment. The flow has two separate phases:

1. Structure. If the repo comes with content, it moves it to the layout and verifies —without running anything— that notebooks, paths and references still resolve. It doesn't audit or validate the code that was already there: it's what's there, tidied up.
2. Materialization. Only then does it implement the mathematics and subject it to the validation ladder.

<repo>/
├── <Name>/            Notebooks/  Data/  Results/  Models/
├── src/<Package>/     one implementation per mathematical object
├── tests/             smoke · invariants · synthetic · audit · remedies
└── pyproject.toml

The ladder has five levels, from the cheapest to the most expensive: smoke, invariants (each claim of the proposal anchored to a test), synthetic (deterministic, fixed seed), audit (findings about the mathematics, measured over 200 random configurations) and remedies (each proposed fix validated with the same rigour). A finding without a validated remedy isn't reported.

That is all that is needed to use the forge. What goes on inside each skill —its steps, its pieces, how they connect to each other and what limitations each one has— is detailed in Anatomy of each skill. For the literal contract, each one's SKILL.md.

Inside the workspace

smith init creates a self-contained directory —with no git initialized and without touching the framework checkout— with this shape:

my-paper/
├── .papersmith/                 # workspace state (config, manifest, ledger)
│   ├── config.json              #   active target and profile, ledger paths
│   ├── manifest.json            #   hashes of everything the framework generated
│   ├── runs_ledger.jsonl        #   every dispatched run, in order
│   └── version                  #   framework version that created it
├── .claude/agents/              # the twenty-seven subagents (single source of truth)
├── skills/                      # the kit copy: the skills + _core
│   └── _core/                   #   the two shared engines (deliberation, implementation)
├── sections/                    # the ten section contracts (they travel WITH content, like skills/)
├── guidance/
│   ├── reference-papers/        # drop-zone: reference PDFs go here
│   ├── paper-guide/             # guiding papers for method and style (optional but recommended)
│   └── data-paper/              # data paper (mandatory for experimental-deliberation)
├── proposals/                   # managed revisions, flat: research-concept-rNN.md
├── paper/                       # the paper's LaTeX (paper-writing writes it here)
├── experiments/                 # the managed experimental protocol
├── implementations/             # destination repos; each one with its own git
├── kaggle-inbox/                # what comes back from the remote workers
├── CLAUDE.md / OPENCODE.md / .agents/AGENTS.md   # harness routing
├── .pi/APPEND_SYSTEM.md         # harness routing (Pi appends it to the system prompt)
├── papersmith.yaml              # workspace configuration (editable)
├── package.json                 # workspace Node dependencies (jiti, typebox), editable
├── requirements.txt
├── scripts/setup_env.py         # ingestion runtime provisioning
└── README.md                    # brief description of the workspace

Every research folder (guidance/*/, proposals/, paper/, experiments/, implementations/) arrives empty, with its own .gitkeep: the workspace's .gitignore applies the same rule this repository applies to itself —the folder is versioned, the content never is. The tutors' threads and the pages you upload to them (.papersmith/math-tutor/, .papersmith/experiments-tutor/) are ignored too: they are yours.

papersmith.yaml — the configuration you will actually touch

The file comes with three example targets and three execution profiles. The targets declare where the work runs; the profiles, what runs and how it is split up:

compute_targets:
  default: "kaggle-gpu-pool"
  targets:
    local-workstation: { provider: "local", device: "cuda:0" }
    kaggle-gpu-pool:   { provider: "kaggle", accelerator: "GPU_T4_X2", ... }
    slurm-cluster:     { provider: "remote-ssh", host: "hpc.university.edu", ... }

execution_profiles:
  smoke_and_invariants:  { target: "local-workstation", entrypoint: "{target_python} -m pytest {target}/tests/test_invariants.py" }
  sweep_training:        { target: "kaggle-gpu-pool", sharding: { … }, artifacts: { … } }
  benchmark_evaluation:  { target: "kaggle-gpu-pool", entrypoint: "papermill …/benchmark.ipynb" }

paper_ingestion:
  engine: auto          # route each page: pypdfium2 fast path, Docling only where needed
  mode: fast            # fast (route pages) | balanced (every page through Docling)
  strip_references: true
  source_base: guidance

The active target and profile live in .papersmith/config.json; papersmith target set changes them. The paper_ingestion block is owned by the ingestion skill and the orchestrator treats it as opaque (it is edited with the skill, not by hand).

What is versioned and what is not

smith upgrade's preservation contract is explicit: it never touches guidance/, proposals/, paper/, experiments/, implementations/, kaggle-inbox/, papersmith.yaml, package.json, README.md nor the .env* files. Everything else is framework scaffolding and upgrade syncs it —with --force if needed— leaving the artifacts intact.

In this repository the same rule holds for Git: the scaffolding is versioned, the content is not. Research folders travel with their .gitkeep and nothing else; guidance/ versions the folders and never the PDFs nor the ingested Markdown. sota-pool/ follows the same contract: the folder travels, and candidates.json, atlas.json and atlas.html —which inlines the full 3D viewer— stay on the machine that produced them.

Preserving is not migrating. Not touching an artifact protects it while its shape does not change; the day a release changes that shape, something is needed to carry it up to it. That is the job of artifact migrations, and it is also the reason .papersmith/version stopped advancing just because files had been copied.

The state: .papersmith/

• manifest.json stores the hash of every generated file. papersmith audit --check-drift compares against the disk and names what drifted; the repair is smith upgrade, never a hand edit.
• runs_ledger.jsonl is append-only: every smith run that gets dispatched is recorded with its target, its profile and its result.
• config.json stores the active target, the active profile and the ledger's configuration paths.
• migrations.json records which version-tied artifact migrations have already been applied, so that upgrade does not repeat them. Convergence ones are never recorded: they never end. If the file gets damaged, upgrade refuses and status reports it; it must not be edited by hand.
• version is the version the workspace declares for its artifacts, not for its files. Of the two markers it is the one that lags behind when a migration fails, and that is why status can say the workspace has not yet reached the installed release.

Harnesses and skill projection

skills/ at the repository root is the single source of truth. Each harness reads skills from its own folder, so the projection script symlinks the canonical tree into each one with relative and idempotent symlinks: running it again converges to the same layout and the checkout stays relocatable.

npm run setup:harnesses      # = bash scripts/setup-harnesses.sh

What each harness wires

smith init and smith upgrade generate, for each harness the workspace declares, what this table shows. It is the same matrix as HARNESS_CAPABILITIES in src/papersmith/generators.py, and a test (tests/test_harness_parity.py) verifies that both match. wired is what gets generated; unsupported is what does not get generated; opt-in is only documented, and it is enabled manually.

When it finishes, init and upgrade print one line per active harness with the state of each capability (wired, unsupported, failed or blocked) and add the wiring key to its result. If a skill symlink could not be created, or there is a real directory where the symlink would go, they report it as a warning (failed / blocked) instead of silently omitting it.

Pi's agents use the web tools web_search and fetch_content from the pi-web-access package. The workspace declares it in .pi/settings.json, which Pi reads once you trust the project and installs from there; nothing needs installing by hand. init writes that file, upgrade adds it to an older workspace that lacks it, and neither touches an existing one (add packages to it with pi install -l <source>).

What the table does not promise:

• Pi: the agents in .pi/agents are only read by the third-party package pi-subagents; the Pi core has no subagents. Without that package, the files exist but remain unused.
• Claude Code: the guard is opt-in. Papersmith never writes or merges .claude/settings.json; the snippet to enable it is in docs/guard-hooks.md.
• Antigravity: there is no guard until its hooks' input schema is obtained (.agents/hooks.json).
• Antigravity, commands: no command files are generated. Its only slash-command mechanism, workflows, is deprecated in favor of skills (retirement 2026-11-01) and the workflows' workspace path is not verified (.agents/workflows is an assumption). Skills already work as /<skill-name> through .agents/skills, so all of them stay invocable as commands.
• Antigravity, documented locations only: .antigravity/rules.md and .antigravity/skills are no longer generated. Antigravity uses .agents/skills, .agents/agents/ and .agents/AGENTS.md (plain text, no frontmatter). Existing workspaces are not migrated: smith init must be run in a new workspace.

Verified support matrix

Summary of docs/harness-support-matrix.md, read against each tool's documentation on 2026-10-02. It is what the tool documents, not what Papersmith generates (that is the table above). unverified means no primary source was read.

• .antigravity/skills and .antigravity/rules.md: unsupported (no source found) and no longer generated; Antigravity uses .agents/skills, .agents/agents/ and .agents/AGENTS.md.
• Antigravity deprecated workflows in favor of skills (2026-11-01, source: https://antigravity.google/docs/ide/workflows).
• Pi: the tool names mcp and mcpScript have no source (unsupported, no source found).
• The docs of these tools change fast; a cell should be re-verified before being relied on after that date.

The sources of each cell are in docs/harness-support-matrix.md.

Inside a generated workspace, the behavior is slightly different: the workspace ships the full skills/ tree and the agent tree (.claude/agents/), and its routing docs point at the canonical tree skills/*/SKILL.md. The routing documents are generated projections: the real source of each agent is .claude/agents/*.md and the source of each skill is its SKILL.md. The projections must not be edited by hand; they regenerate (and smith audit --check-drift warns if one drifted). init and upgrade generate one slash command per skill in .claude/commands/, .opencode/commands/ and .pi/prompts/; Antigravity does not receive command files and reads skills through its symlinks. Agents are projected from .claude/agents/ to .opencode/agents/, .pi/agents/ and .agents/agents/. For a harness to list the skills as / commands, npm run setup:harnesses must be run inside the workspace: it symlinks the shipped tree into .claude/skills, .pi/skills, .opencode/skills and .agents/skills, just as relative and idempotent as in the checkout. In this checkout, the slash command projections (.claude/commands/, .opencode/commands/, .pi/prompts/), the agent projections (.opencode/agents/, .pi/agents/, .agents/agents/) and the OpenCode and Pi guards are synced with python scripts/sync-repo-harness.py (--check for CI; it has no script in package.json because the kit ships it and the script is repo-only).

The agents

A workspace ships twenty-seven subagents in .claude/agents/. They are short bursts of work that a harness launches (for example, with the Task tool) on request, not replacements for the CLI. The shape is always the same — one burst between two operator gates: the agent decides nothing, asks nothing, and ends up reporting what it found and how much it cost to establish it. The operator opens and closes the gates.

Three rules that order everything else:

• The source of truth is .claude/agents/. Each harness's routing docs (CLAUDE.md, .pi/APPEND_SYSTEM.md, OPENCODE.md, .agents/AGENTS.md) list all twenty-seven, and they are generated: an agent is changed there, not in the projection.
• Each agent declares the skill it loads (skills/<name>/SKILL.md), and that binding is verified: an agent that points at a skill the workspace does not ship does not get there.
• Some agents are not invoked by any code path (contract-auditor, redactor, style-sampler): they are launched manually, and it is the write verb that judges what they return — it never trusts the operator's account nor the draft on their own.

Which model runs each agent. An agent with no model: line runs on the session's model. Three agents only read (Read, Glob, Grep) and hold no gate — math-tutor, insumos-observer and style-sampler — so they carry model: sonnet. Claude Code reads that line straight from .claude/agents/, because the workspace copies that directory rather than rendering it. The projections for the other harnesses drop the line. Pi has no cited per-agent model field; OpenCode (provider/model) and Antigravity (inherit, flash, pro) have one, but nothing cited says which of their values matches sonnet, so writing one would be a guess. On those three harnesses every agent runs on the session's model.

Agents read only the stage they are in. proposal-implementation keeps each probe state's section in its own file, skills/proposal-implementation/references/next-step-<nextStep>.md, and its SKILL.md leaves a Stage reference: line where the section stood. implementation-build, implementation-walk and paper-replicator read SKILL.md, then only the reference for the nextStep that probe reports.

Anatomy of each skill

The guided tour of every skill, with its flow diagram, is in docs/skills.md. The workspace ships nineteen skills; each one's SKILL.md is its source of truth:

• datasets
• experiment-diagrams
• experimental-deliberation
• experimental-implementation
• experiments-tutor
• figure-review
• hyperparameter-search
• journal-finder
• kaggle-accounts
• math-tutor
• paper-ingestion
• paper-writing
• plausibility
• proposal-deliberation
• proposal-implementation
• remote-execution
• remote-parallelism
• skill-audit
• upgrade-project

The CLI commands

Every smith command, with its options and examples, is in docs/cli.md; smith <command> --help is the source of truth.

Remote compute, carefully

Before a job touches a real worker, the forge offers a ladder of cheap rehearsals, worked down in order and not skipped:

1. smith target list — which targets exist and which one is active.
2. smith target check [<name>] — does that target answer?
3. smith run <profile> --dry-run — plans and records the command in the ledger; dispatches nothing.
4. smith remote … --smoke — a real rehearsal against the backend, without spending a campaign.
5. Only then, the real run (--consent <token> where applicable).

The targets and profiles are those of the workspace's papersmith.yaml (see Inside the workspace). The ledger .papersmith/runs_ledger.jsonl is append-only: every dispatched run is recorded with its target, its profile and its outcome. And smith remote status / pull is the memory of what was sent out and what came back — the detail of that contract lives in the remote-execution section of the Anatomy.

Credentials and backends. remote-execution supports several backends (kaggle, colab):

• Kaggle: runs against the configured worker pool. Its credentials live in the kaggle-accounts store (skills/kaggle-accounts/store/accounts.json), never leave the disk and are never printed: materialize writes a worker's token to a text file and reports where, never what. /kaggle-accounts validates or removes accounts, and smith target check kaggle tests them from the side. A papersmith accounts subcommand does not exist yet (recorded follow-up): it is invoked as python3 skills/kaggle-accounts/scripts/accounts_cli.py {list,discover,validate,remove,materialize}; validate --account <username> --json re-checks one stored account and writes nothing, which is how preflight checks each worker.
• Colab: runs against sessions managed by google-colab-cli (smith remote push --backend colab ...). It is not a workspace target (it is not passed to init --remote or to target check), but a direct backend of remote-execution. It requires google-colab-cli on PATH and the token resolved in ~/.config/colab-cli/token.json.

Jobs checked before they run. remote_cli.py generate-jobs writes one job folder per remote step a product declares in __steps__ (generate-job --dataset <slug> and generate-jobs --dataset declare a workspace dataset as the job's input, uploading nothing), and remote_cli.py preflight checks every generated job locally before its first remote rehearsal: pin, diagrams, interpreter, code reachability, inputs, a local rehearsal of the smoke, each account, the accelerator and session fit, one pass/warn/fail row each. It never submits.

Placement by measurement. Before a proof of concept or a full experiment runs, remote-parallelism's timing probe measures each candidate way to run it (local CPU with N processes, a local accelerator, each account's machine, a CPU-only remote session, batch and loader settings), and plan recommends the fastest affordable one with its reason. A GPU is never the default: it is chosen only when measured faster, and generate-job --accelerator-kind cpu writes a CPU-only remote job when the plan says so.

Runs longer than one session. A run whose estimate exceeds a service's continuous session is planned as a chain of sessions that checkpoint every epoch or trial and resume: plan --resumable and schedule cost the chain, with the setup paid again in every session and the wait for each weekly quota reset counted (quotaWeeks). preflight --unit <id> ... rehearses it locally before the first remote rehearsal: the job must stop at a deadline that has already arrived, leave partial.json and its checkpoints, and finish only the remaining units in a second session. submit --unit then carries the declared checkpoints to each continuation on the same account (Kaggle mounts the earlier output; Colab uploads it into the new session). The rule lives in remote-execution's "The session-chain rule".

Campaigns larger than the pool. distribute and campaign submit place every unit: past the granted slots, units join per-slot queues that each job runs in order, cut into chained sessions on the same account when a timing record says so, and schedule and plan report the same queues. Units already returned complete (done) or still running (inFlight) are never placed again, so the campaign's full unit list can be passed every time, and remaining names the units still owed.

Real quota. A real submission spends quota and GPU hours. The house rule is simple: everything that can be rehearsed with --dry-run and --smoke is rehearsed; what is left after that is a decision for the operator.

Architecture

papersmith-ai/
├── docs/                       # supporting documentation (MCP server)
├── src/papersmith/             # the orchestrator CLI (Python) and its runtime bridges
│   ├── core/                   # init, status, ingest, upgrade, executor, ledger
│   ├── bridges/                # Node, Python, deliberation, remote execution
│   └── mcp/                    # stdio Model Context Protocol server
├── skills/                     # canonical skill tree, projected to the harnesses
│   ├── _core/                  # shared deliberation and implementation engines
│   ├── paper-ingestion/        # Marker + Surya OCR + llama-server
│   ├── proposal-deliberation/  # TypeScript engine: AST verification and state machine
│   ├── proposal-implementation/# reproducible code generation and its tests
│   ├── experimental-deliberation/   # second domain; same engine, a different aim
│   ├── experimental-implementation/ # second implementation host; two documents, one engine
│   ├── paper-writing/          # section contracts → PDF, assuming nothing
│   ├── remote-execution/       # dispatch to distributed compute (Kaggle T4/P100 / local)
│   ├── kaggle-accounts/        # credentials gatekeeper: identities, never values
│   └── skill-audit/            # meta-auditor of surfaces and promises
├── guidance/                   # reference papers and domain guidelines
├── tests/                      # Node.js + pytest suites (meta-audits included)
└── scripts/                    # environment provisioning and harness setup

smith team {list,add,remove} and smith colab {login,status}

smith team add --owner ana --name "Ana" --kaggle-username ana_k --colab --colab-tier pro --attest
smith team list
smith colab login --owner ana
smith colab status

Several remote accounts are supported as a team: one account per person per service. team add records an owner in .papersmith/team.json (git-ignored, no secrets) and refuses without --attest, the person's confirmation that they own and operate exactly one account per listed service, and refuses an account already listed under someone else. Google's Colab FAQ disallows "using multiple accounts to work around access or resource usage restrictions", and on free runtimes without compute units "running distributed computing workers"; Kaggle's terms could not be read, so the same rule applies. A Colab owner on the free tier may run interactive or proof-of-concept work in one session, never a worker of a split campaign.

colab login --owner <slug> runs the official google-colab-cli (pinned 0.6.0) consent for that owner under a private HOME of its own (~/.config/papersmith/colab/<owner>/, mode 0700): the CLI keeps its token under HOME, so a private HOME is what isolates one person's account from another's, and papersmith never opens a file inside it. colab status reports, per owner, the CLI version, whether the sign-in still answers, the tier and whether the owner may take part in a split campaign. smith init and smith upgrade also register Google's official Colab MCP server (colab), which drives a notebook the person has open in the browser.

Engineering guarantees

• AST invariants and boundaries: equations and sections are tracked by stable IDs and immutable revision chains. There are no unauthorized deletions, relocations or boundary merges.
• Keyless and local-first: ingestion runs fully offline and locally, with Marker and llama.cpp.
• Harness-agnostic projection: the canonical skills/ tree is projected to .claude/skills, .pi/skills, .opencode/skills and .agents/skills.

Development and testing

There are two suites, not one. Running a single one and giving a verdict is a mistake this repository has already made, so both are here with their exact invocation:

npm test                                  # shared deliberation engine
.venv/bin/python -m pytest                # everything else
npm run test:all                          # both, in order

A fast loop, not a gate. npm run test:fast (python3 scripts/fast_tests.py) runs the tests that touch the uncommitted work (staged, unstaged and untracked; non-code files such as .md or .json select nothing) plus a fixed set of guards. The guards alone take about 40 seconds and the selection adds to that. Three measured slow tests are deselected by default (--with-slow includes them; the full gate always runs them). --since REF adds what is already committed since REF, --list shows the selection, its base and why, and --guards-only runs the guards alone. It is a textual heuristic that can miss tests exercising a module indirectly: it does not replace npm run test:all, which remains what is run before integrating.

Python in parallel. npm run test:py:par runs the same suite with pytest -n 8 --dist loadfile (it needs pytest-xdist, which setup_env.py already provisions). Measured on 2026-10-01: 5455 passed in 5 min 9 s against 10 min 17 s in series. Only loadfile is safe: load and loadscope compete for the shared implementations/ directory. A single test (waiting for a killed process, 120 s) sets the time floor.

npm run typecheck runs tsc over the deliberation engine and comes out clean.

Two traps that look like repository defects and are not.

The npm script sets the domain profile. npm test exports DELIBERATION_DOMAIN_PROFILE before invoking node --test. Calling bare node --test fails closed, because the engine serves no domain of its own and refuses to start without one. It is the invocation that is wrong, not the engine. The same holds on the Python side: the implementation engines require IMPLEMENTATION_DOMAIN_PROFILE, which is why the CLI that launches them sets it — and why tests/domain_profile.py seeds and restores it by import.

tests/ has no __init__.py. The form one would write from memory, discover -s tests -t ., dies with ImportError: Start directory is not importable. The form that works is the one above, without -t ..

What tests what. tests/ covers the forge's tooling: the two shared engines, every skill's SKILL.md and its scripts, the tie between each skill and its agents in both directions, and a from-scratch series that generates a workspace with the CLI and walks every command, every embedded skill and every subagent inside it (tests/test_workspace_{commands,skills,agents}_e2e.py). The tests of a materialized implementation live in its destination repository and nowhere else; deleting it deletes its tests.

One live rehearsal, opt-in. Everything above is hermetic. To exercise the real engines — npm install, environment provisioning, an ingestion and an agent — there is scripts/cli-paper-live-smoke.sh, which refuses to run without PAPERSMITH_LIVE=1 (and accepts PAPERSMITH_LIVE_INGEST=<pdf> and PAPERSMITH_LIVE_AGENT=<name> to extend the walk).

Two sealed corpora. tests/seal/ and tests/experiments_seal/ keep the exact output of a battery of commands as golden digests. They are not unit tests: they exist so that a change in the shared engine cannot silently alter what each host produces. They are worth what they cost — during the last fix, four digests moved and said the correction was wrong, in a direction no unit test was seeing.

proposal-implementation was also validated with its own scenario harness — the situations a clone can fall into, plus the full cycle up to publishing the revision that produces a finding — built to close v1.5 and retired when it closed. It was scaffolding, not part of the forge. It stays in the history: the commit that removes it says so, and git revert brings it back if it ever has to run again.

Troubleshooting

"The skill refused to start." That is the expected behaviour: each skill declares what it needs first and exits with a refusal code that names what is missing, rather than inventing it or skipping it. The code reads as a to-do list — it says exactly which input is missing. It is not a mistake on the reader's part.

"Bare node --test fails." By design: the deliberation engine refuses to serve without a domain profile. npm test sets it, and so does the skill's CLI. The same holds for the implementation engines: IMPLEMENTATION_DOMAIN_PROFILE_REQUIRED means they are being invoked directly instead of through implementation_cli.py, which is what starts them with their profile. If you invoke them directly on purpose, set the variable to an absolute path: a relative one is refused with IMPLEMENTATION_DOMAIN_PROFILE_NOT_ABSOLUTE.

"The workspace has no .claude/skills." That is the default state: a workspace ships the whole skills/ tree and .claude/agents/, and its routing documents point at the canonical skills/*/SKILL.md tree; the agents read the skills from there. For a harness to list them as / commands, run npm run setup:harnesses inside the workspace: it creates the projections of the four harnesses, relative and idempotent.

"smith audit reports drift." Some generated file changed with respect to the manifest. It is not edited by hand: run smith upgrade (and --force if needed) to return to the framework's version.

"papersmith accounts cannot be run." It does not exist yet (recorded follow-up). Use python3 skills/kaggle-accounts/scripts/accounts_cli.py {list,discover,validate,remove,materialize}.

"status exits 0 even when there is drift." A known, recorded gap: the exit code is not strict today. For a CI gate, read the JSON (--json) rather than the exit code.

"The first ingestion downloads ~1.5 GB." Those are the Surya weights when the environment was not pre-provisioned. smith init already provisions the environment unless --no-env was passed; in that case, run python3 scripts/setup_env.py install before the first ingestion.

Related documentation

• Documentation index, and where history lives: docs/README.md.
• Every skill, guided: docs/skills.md. Every command: docs/cli.md.
• Each skill, its literal contract: skills/<name>/SKILL.md — the source of truth for what a skill accepts and promises.
• Project context for agents: openspec/project-context.md.
• Domain guidelines: guidance/paper-guide/ — the drop-zone that proposal-deliberation loads as context at the start of each deliberation.
• Environment and harness provisioning: scripts/setup_env.py and scripts/setup-harnesses.sh.
• Suites and gates: tests/ and the package.json scripts (test, test:all, typecheck).
• MCP server and tool catalogue: docs/mcp.md.
• Flow diagram (commands, skills and agents in Pi): docs/diagrams/papersmith-pi-flow.html.

License

Apache-2.0 — the full text is in LICENSE.