Skip to main content

run-harness

run-harness launches one coding-agent harness (Claude Code, Codex, Pi or OpenCode) non-interactively, through one set of options, and prints the harness's final message on stdout. It builds the command line, starts the harness, extracts the answer and maps the exit code. It is not an agent runtime, and it never reads or writes harness config files.

Callers are mostly other programs that run it as a subprocess, plus people at a terminal. There is no Python library API. The detailed contract is in docs/spec.md.

Supported: Linux and macOS, Python 3.12+. Windows is not supported.

Install and use

uvx run-harness claude "Summarise the architecture of this repo"
uvx run-harness codex "List the public functions in src/" -m gpt-5-codex -e high
uvx run-harness pi "Where is the config parsed?" -C ~/code/project
uvx run-harness opencode "Explain the build" -m anthropic/claude-sonnet-4-5 -e high

You can also install it with uv tool install run-harness. The harness itself (claude, codex, pi, opencode) must already be installed, logged in and on PATH. run-harness passes the environment and auth through untouched.

run-harness <harness> [PROMPT] [-m MODEL] [-e {low,medium,high,max}] [-f PATH] [-C DIR] [--autonomous] [-- PASSTHROUGH...]
run-harness --version | --help
run-harness <harness> --help

Only the final message goes to stdout, with a trailing newline added if it is missing. Progress and diagnostics go to stderr.

option meaning
PROMPT the prompt text, or - to read the prompt from run-harness's own stdin
-f, --prompt-file PATH read the prompt from a file
-m, --model MODEL model, passed verbatim and never validated
-e, --effort LEVEL low, medium, high or max. If omitted, the harness default applies
-C, --cwd DIR directory the harness runs in (default: the current directory)
--autonomous lift the read-only restrictions
-- … passthrough arguments, handed to the harness verbatim

Options may come before or after PROMPT. An unknown option before -- is an error (exit 2). Abbreviated long options are not accepted. --model "" is an error.

Prompt source

Give exactly one of: PROMPT, PROMPT = - (read stdin), or -f PATH. None, or more than one, is exit 2. Empty or whitespace-only prompts and prompts that are not valid UTF-8 are exit 2. A Claude Code prompt over 10 MiB is rejected. No other harness limit is checked.

  • run-harness reads its own stdin only when PROMPT is -. -f - is rejected.
  • -f PATH is resolved against the directory you ran run-harness from, not --cwd. FIFOs and process substitution (-f <(...)) work. Directories are rejected.
  • The prompt reaches the harness on its stdin, never as an argv argument. It is passed verbatim.
  • A prompt starting with - cannot be given as PROMPT. Send untrusted text, and anything that starts with -, through -f or stdin:
printf '%s' "$UNTRUSTED" | uvx run-harness claude -
uvx run-harness codex -f prompt.md

Working directory

--cwd must exist and be a directory. It is made absolute without resolving symlinks, and it is applied both as the harness's working directory and as PWD in its environment. No harness gets its own directory flag (codex -C, opencode --dir).

Passthrough

Everything after the first -- is appended after the arguments run-harness generates, and is never inspected:

uvx run-harness claude "Review this" -- --max-turns 5

Conflicts with the normalized options are your responsibility. See "do not override" below. If an override breaks final-message extraction, the run exits 1.

Read-only and autonomous

Read-only is the default. The model may read and reason, but cannot change files or reach the network. It may run a command only where a sandbox stops that command from writing (Codex). --autonomous lifts that and permits writes, command execution and network access, in whatever way each harness does it (see the table below).

The limits matter:

  • run-harness only sets the harness's own flags. It does not sandbox anything itself.
  • Hooks and extensions still run. User and project hooks run in Claude Code's read-only mode, and Pi extensions still load and run their hooks. Only the tools are restricted.
  • User deny rules still apply under --autonomous (Claude Code deny settings, deny rules in the OpenCode config).
  • Codex read-only still lets the model run commands. They run inside a sandbox that blocks writes and network access.
  • Pi has no permission system. Its read-only mode is a tool allowlist.
  • A permission that is blocked in read-only mode is not an error. OpenCode ends such a run with exit 0.
  • Org, managed or MDM OpenCode config merges after the config run-harness injects.

Per-harness mapping

Claude Code Codex Pi OpenCode
base command claude -p --output-format stream-json --verbose codex exec --skip-git-repo-check --color never pi --mode json opencode run --standalone --format json
prompt stdin stdin stdin stdin
--model M --model M -m M --model M -m M
--effort L --effort L -c model_reasoning_effort=L --thinking L -m M#L (pre-checked)
read-only (default) --tools Read,Grep,Glob --permission-mode dontAsk --strict-mcp-config -s read-only --tools read,grep,find,ls (plus --no-mcp on Pi 1.x and later) injected agent plus --agent run-harness-readonly (pre-checked with opencode api)
--autonomous --dangerously-skip-permissions --dangerously-bypass-approvals-and-sandbox nothing added --auto
final message result of the last result event captured stdout text of the last assistant message_end text parts of the last assistant message
failure despite exit 0 is_error: true, or no result none stopReason is error or aborted an error event
do not override -p, --output-format, --verbose, --tools, --permission-mode -s, --color, --json, -o --mode, --tools, -p --format, --agent, --dir

Argument order is: base command, permission arguments, --model, effort, passthrough.

Claude Code

  • CLAUDE_CODE_EFFORT_LEVEL passes through untouched. Whether it or --effort wins is unverified.
  • A user maxEffortLevel setting can silently cap max.
  • Read-only uses a tool allowlist, not plan mode, and --strict-mcp-config drops MCP servers from user and project config. Hooks still run.
  • Bypass mode refuses to run as root outside a sandbox. That refusal comes through with Claude's exit code.
  • An auth failure (Not logged in) exits 1 with the message on stderr.

Codex

  • Codex's stdout, when piped, is the final message. It is buffered and written once at the end. Its stderr is its own progress, forwarded unchanged.
  • -s read-only is always passed explicitly. Projects marked trusted in ~/.codex/config.toml would otherwise default to workspace-write.
  • An unknown model makes Codex warn ("fallback metadata") and fail the turn, so the run exits 1.
  • There is no auth pre-check. An auth failure exits 1 after roughly 15 to 20 seconds of retries.

Pi

  • Before a read-only run, run-harness runs pi --version. Pi 1.x and later gets --no-mcp, because --tools keeps MCP tools otherwise. Pi 0.x does not get it, since the flag does not exist there. Unparseable output or a non-zero exit is exit 2, because read-only cannot be guaranteed. --autonomous skips the check.
  • --thinking is clamped silently per model.
  • A :<thinking> suffix on --model is overridden by an explicit --effort.
  • Extensions still load and run their hooks in read-only mode. Only their tools are hidden.
  • Passthrough positional arguments starting with @ become file attachments.
  • Pi exits 0 when the model errors in JSON mode. run-harness turns that into exit 1.

OpenCode

  • Read-only injects an agent named run-harness-readonly through OPENCODE_CONFIG_CONTENT. It is merged into your value if you set one, and invalid JSON there is exit 2. The agent allows only read, glob, grep, list and lsp, and denies everything else, including task. Nothing is persisted. Apart from PWD, this is the only environment variable run-harness changes.
  • Supports OpenCode 2.x only. Every run passes --standalone: 2.x otherwise talks to a shared background service that never sees the injected config, so the agent would be missing.
  • Before launch, opencode api --standalone GET /api/config (same env and cwd) must show the agent with exactly the injected permissions (otherwise exit 2). If a run still cannot load the agent, OpenCode reports Agent not found and run-harness exits non-zero with nothing on stdout.
  • --effort requires --model in provider/model form. run-harness passes it as -m provider/model#level and first reads opencode api --standalone GET /api/model. It exits 2 if the model is not listed, has no variants, or the level is not one of its variants.
  • The model is validated only when --effort is given. Without --effort, an unlisted model fails at run time, and the run exits 1.
  • A permission rejected in read-only mode ends the run with exit 0.
  • OpenCode creates .opencode/.gitignore and installs its plugin package at startup. Those are its own side effects.

Effort

low, medium, high and max map one to one onto every harness, with no remapping (max is never turned into xhigh). Claude Code, Codex and Pi adjust a level the model does not support to the nearest one it does, and run-harness does not detect that. OpenCode rejects unknown variants at run time, but run-harness checks them first so you get exit 2 and the valid levels. Native-only levels (xhigh, minimal, off, none and so on) are reachable only through passthrough.

Exit codes

situation exit stdout
usage or validation error, or a failed pre-launch check 2 empty
harness executable (or a pre-check helper) not on PATH 127 empty
harness exits non-zero the harness's code, verbatim empty
harness killed by signal N 128 + N empty
harness exits 0, but the event stream reports an error 1 empty
harness exits 0 with no final-message event 1 empty
harness exits 0 with a final message 0 the final message
harness exits 0 with an empty final message 0 empty (no newline added)
Codex exits 0 0 the captured stdout

stdout is written once, after the harness exits, and only when run-harness exits 0. On failure, error text taken from the stream goes to stderr.

stderr

stderr is for people. Its format is not stable and is not a contract. The harness's own stderr is forwarded unchanged. For Claude Code, Pi and OpenCode, run-harness also prints the assistant text and a → tool: <name> line per tool call as events arrive. Lines that are not JSON are forwarded raw. Codex has no renderer, since its stderr already is its progress.

Tested harness versions

run-harness declares tested versions, not minimums. The fixtures come from these versions run against a local fake model server (OpenCode: observed on 2.0.32 with a free model), plus hand-written edge cases. Claude Code's and OpenCode's success and tool_call fixtures, and Claude Code's not_logged_in, are recorded against a real model with the recorder script below. Codex was checked with the live suite:

harness version
Claude Code 2.1.303
Codex codex-cli 0.163.0
Pi 0.85.1
OpenCode 2.0.32 (2.x only)

There are no general runtime version checks. The one exception is Pi's --no-mcp, because a version difference there would make read-only fail open. Other versions may work, or may break extraction. The opt-in live suite (below) is how drift is detected.

Development

uv run pytest                       # unit and fake-harness tests; no real harness needed
uv run ruff check
uv run ruff format --check
uv run pytest -m live               # opt-in smoke run against the real harnesses on PATH
uv run scripts/record_fixtures.py   # re-record the fixtures from the real harnesses

The live suite never runs in CI. Run it locally before a release, and re-record the fixtures if it fails.

Release

Releases run from the Actions tab: run the Release workflow on master, choosing a version bump (none releases the version already in pyproject.toml) and whether to publish to pypi.

The workflow runs lint and tests, applies the bump and builds, refusing a version that is already tagged. It publishes to TestPyPI and smoke-tests the install on Linux and macOS (scripts/smoke_release.sh). With pypi ticked it then publishes the same files to PyPI, commits the bump, tags vX.Y.Z, creates a GitHub Release with generated notes and smoke-tests the PyPI install. With pypi unticked it stops after TestPyPI, as a dry run. Both indexes use trusted publishing for release.yml, with no environment.

Metadata

Release files for run-harness 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for run-harness 0.1.0
File Size Uploaded Provenance Actions
run_harness-0.1.0.tar.gz 16.8 kB Without provenance Details for run_harness-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for run-harness 0.1.0
File Interpreter ABI Platform Actions
run_harness-0.1.0-py3-none-any.whl Python 3 none any Details for run_harness-0.1.0-py3-none-any.whl

Total release size: 38.0 kB

Release files / run_harness-0.1.0.tar.gz

Download URL run_harness-0.1.0.tar.gz
Size 16.8 kB
Tags Source
SHA-256 checksum
How to use checksums
348aacf124565dcabed76972d28332313092e73f3ff0a41dc322d79fbba8c6e8
BLAKE2b-256 checksum
How to use checksums
2dcd8ee1180dc4c08825232e42b531d6218348ccb16bef1092d637909fd32838
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.6 {"installer":{"name":"uv","version":"0.13.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / run_harness-0.1.0-py3-none-any.whl

Download URL run_harness-0.1.0-py3-none-any.whl
Size 21.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b02da2550dad9679daa82fbdd5d4cd85250515dfb3bc7fa6c53203bed9db3a25
BLAKE2b-256 checksum
How to use checksums
1866753b7a7dadcf1285e71da045726a16008e2dcc4bca176c036b54a0a5239f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.6 {"installer":{"name":"uv","version":"0.13.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for run-harness 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page