Orchestra Review
Your coding agent keeps making the same mistakes, because every session starts with no memory of your machine or its tools. It calls a program that isn't installed, imports a module from the wrong Python, or trips over the same shell quirk, week after week.
Orchestra Review reads the session logs that Claude Code, Codex CLI and OpenCode already keep on your computer. It finds the mistakes that recur across sessions and fixes each with one instruction line. It can also block a known mistake before it runs, and it measures whether a fix actually worked. Nothing leaves your machine.
pipx install orchestra-review
orchestra-review
What it found
In 67,074 public agent runs (OpenHands with Qwen3-Coder on real GitHub issues from 1,823 repositories):
- 41% of the failures with an identifiable cause repeated a mistake seen in another repository.
- 62% of runs hit at least one of those mistakes.
- The largest was asking to view lines past the end of a file: 21,780 runs across 1,509 repositories.
- Second was piping after
find -exec ... \;, which that shell tool rejects: 5,406 runs. - Built-in rules cover 59% of those repeated failures, and the nine rules mined from this data ship with the tool.
- Runs that hit a repeated mistake resolved their issue less often, 44.5% against 53.6%. That's a correlation, not a measured effect.
Details and limits: research/.
In the author's own logs (as of 2026-11-16): 74 sessions across Claude Code, Codex and OpenCode, with 188 failed tool calls. 51 of those failures (27%) repeated a mistake from an earlier session. The mistakes were all about the machine:
pdftoppmmissing for PDF reads (8 sessions);- zsh treating
echo ====as a command (7 sessions); No module named 'fitz'(7 sessions, across Claude Code and Codex);- zsh erroring on a glob that matches nothing (5 sessions).
Six lines went into ~/.claude/CLAUDE.md on 2026-11-13. Each line is judged only on sessions that tried the action it covers:
| Fix | Sessions that hit it / tried it, before | After | Verdict |
|---|---|---|---|
| zsh: a glob that matches nothing is an error | 5 / 15 | 0 / 7 | stopped |
zsh: quote words starting with = |
5 / 14 | 2 / 4 | too early, still happening |
pdftoppm missing for PDF reads |
8 / 9 | 0 / 0 | too early |
fitz, pypdf, docx not importable |
5 / 10, 2 / 10, 2 / 4 | 0 / 0 | too early |
"Too early" means fewer than five sessions have tried the action since the line was added. One person's sessions are a small sample.
Usage
orchestra-review # what your agents keep getting wrong, with a fix line for each
orchestra-review apply 6eab0c6e --to ~/.claude/CLAUDE.md # add one line, with a timed marker
orchestra-review sync # add rule-based lines for mistakes in 3+ sessions, retire stale ones
orchestra-review undo 6eab0c6e # remove a line; it will never be added back automatically
orchestra-review hook install # Claude Code: block known mistakes live, and sync when a session ends
orchestra-review --public # scrub paths, file names, URLs and emails, for screenshots
orchestra-review --json # everything, as JSON
How it works
-
Finding mistakes. A recurring mistake is a failure in two or more sessions that shares the same last error line, ignoring numbers and ids. Some failures are left out:
- test runs that fail, because the agent is checking its work;
- calls refused by a permission rule or interrupted by you, which are counted as blocked;
- error lines that say nothing about the cause, such as a bare exit code or a path.
-
Fix lines. Lines come from built-in rules: missing programs and Python modules, zsh
=and glob quirks, old-Python syntax, and misused file-viewing and editing tools. For other causes, a local model you run yourself (Ollama,--model) can draft a line. A draft is kept only if it names the cause and contains no URL or command that downloads or deletes anything, because log text is untrusted and the line ends up in a file your agent obeys. Drafted lines are never added automatically. -
sync. It adds a rule-based line once a mistake has hit three sessions. If the mistake was seen in only one project, the line goes into that project's instruction file, but only if the file already exists; otherwise it goes into the agent's global file. Before keeping a line,syncre-checks its fact on this machine: is the program still missing, can the module still not be imported, is the shell still zsh? When a fact stops being true, for example after you installpoppler, the line is removed and logged. A line you delete by hand, or remove withundo, is never added back. -
The hook.
hook installadds two Claude Code hooks to~/.claude/settings.json, keeping a.bakcopy of the original:- Before a tool call that would repeat a known mistake (running a missing program, an unquoted zsh
=word, or reading a PDF withoutpdftoppm), the hook re-checks the fact and blocks the call with the fix as the reason. The agent corrects course without the failed step. - Never blocked:
- missing Python modules, because importability depends on the working directory and virtualenv (a replay of 67k public runs showed most such blocks would have been wrong; see
research/); - zsh glob mistakes, because whether a glob matches depends on the directory at run time;
- commands that install software or change PATH themselves.
- missing Python modules, because importability depends on the working directory and virtualenv (a replay of 67k public runs showed most such blocks would have been wrong; see
- When a session ends, the other hook runs
sync.
Any error in either hook lets the call through.
- Before a tool call that would repeat a known mistake (running a missing program, an unquoted zsh
-
Measurement. Before-and-after counts only sessions that tried the relevant action, so a change of task can't look like a cure.
-
Privacy. Everything is read locally. Nothing is sent anywhere unless you point
--modelat a model server.--publicandsharedrop paths, file names, project names and commands.
Scope
-
Reads:
- Claude Code,
~/.claude/projects, including subagents; - Codex CLI,
~/.codex/sessions; - OpenCode, through
opencode session export, for a project directory.
Cursor is not supported.
- Claude Code,
-
Runs on: macOS and Linux with Python 3.11 or later, with no dependencies. Windows is untested.
-
Maintained: best-effort. Issues are welcome; there are no promised response times.
Changes: CHANGELOG.md. Design notes: docs/design.md. MIT licensed.
Metadata
Release files for orchestra-review 0.2.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| orchestra_review-0.2.1.tar.gz | 42.3 kB | With provenance | Details for orchestra_review-0.2.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| orchestra_review-0.2.1-py3-none-any.whl | Python 3 | none | any | Details for orchestra_review-0.2.1-py3-none-any.whl |
Total release size: 77.4 kB
Release files / orchestra_review-0.2.1.tar.gz
| Download URL | orchestra_review-0.2.1.tar.gz |
|---|---|
| Size | 42.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6f6d98ffbaf5b9695c69405711b6cc007e3ec90283d5abd2e1e31315fa5676d5
|
|
BLAKE2b-256 checksum How to use checksums |
0a37cadce9a24d86b049ab7f66700f897c772de682c777ccf9afd42823a0174c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.
Transparency logRelease files / orchestra_review-0.2.1-py3-none-any.whl
| Download URL | orchestra_review-0.2.1-py3-none-any.whl |
|---|---|
| Size | 35.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f57411c3f09e725552b00ab75877cef8c6994f05c33099951b08567daaf56e2f
|
|
BLAKE2b-256 checksum How to use checksums |
1c1a200e4104eea433fece8339f55b524a201fdad02e415e457e7d2010a9515d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.
Transparency logSecurity for orchestra-review 0.2.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for orchestra_review-0.2.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.