Nookku

(நோக்கு - to observe)

A test harness for developers of agent apps.

Test your chat agent through Claude Code or Codex. The harness model does not retype one message or one reply.

Nookku is a test harness for developers of agent apps. You talk to your agent in the coding harness where you already work. The relay sends each message to your agent byte for byte, and shows each reply byte for byte. The model does not run while you talk. At the end, the model reads the exact record and evaluates your agent: business logic, tone, accuracy.

It adapts to your app through a thin entry in .nookku/, and your app's code does not change. The cost is plumbing: you write the entry once, and you change it when the start or the wiring of your app changes.

How it works

tester ──> harness ──> relay ──────> tap ──stdin/stdout──> entry ──> your app
                         │            │
                         ▼            ▼
                    relay.jsonl    tap.jsonl
            (what you typed      (what the agent
             and saw)             received and sent)
                         │            │
                         └─> audit <──┘   exit 0 clean, 1 break, 2 invalid record

• Harness: Claude Code or Codex, where you type your test messages.
• Relay: the command nookku hook, which the plugin or the project hooks run in the harness. It carries each message and each reply, and the model writes neither.
• Tap: a proxy in front of your agent. It forwards each byte with no change, and records what the agent received and sent.
• Entry: a thin wrapper that starts your app and speaks one JSON line in and one JSON line out.
• Audit: a program that compares the two records byte for byte, and names each break.

docs/architecture.md shows this flow with each part, and has diagrams of one turn, the test lifecycle and the trace.

Two processes write two records, and the audit compares them byte for byte. It reports 8 break classes (SPEC.md section 3.3). It fails closed: it never reports clean on a record that it cannot read. 62 audit cases, 20 contract cases, 18 trace cases, 11 seal cases and 7 receiver cases in conformance/ test the spec (cases, tests).

A test also records what your app did. It copies the session files of the app's own Agent SDK or Codex sessions. It records the backend calls, the direct model calls and the OpenTelemetry spans of the app. docs/architecture.md shows each part.

Install

uv tool install nookku

It needs Python 3.10 or later, and it has no runtime dependencies (pyproject.toml). The docs are for version 0.4.0. If you use verbatim-relay 0.3.x or earlier, do the 0.4.0 steps of the upgrade guide, because the package has a new name (CHANGELOG.md). To upgrade from a later version of nookku, end each running test first. Then run uv tool upgrade nookku. docs/how-to/upgrade.md gives each step.

Quick start

Select one route: try the toy shop, connect your app, or choose a relay.

Try the toy shop

In a clone of this repo, with the toy shop agent and the project hooks in Claude Code:

nookku init claude-code --entry "python3 examples/toy-shop/agent.py"
nookku check        # one message through the entry: PASS proves the connection, not the reply
nookku view         # in a second terminal: each reply shows here

In Claude Code, type the prompt nookku start, then your test messages, then nookku end. The end of the test writes audit.json in the test folder .nookku/tests/<test-id>/. Then the model evaluates the test and writes report.md. docs/getting-started.md shows each step with its real output.

Optional: to see the audit again, run it by hand:

nookku audit --tap .nookku/tests/<test-id>/tap.jsonl --relay .nookku/tests/<test-id>/relay.jsonl

Connect your app

In the harness, in the project folder of your app, type !nookku setup. Then type the prompt "Follow the Nookku setup guide, and connect a test to this app." docs/how-to/test-your-app.md gives each step from the install to the results, with the expected output. To write the entry yourself, read docs/how-to/connect-your-agent.md.

Choose a relay

You install the relay as the plugin or as the project hooks. Both work in Claude Code and in Codex, with the same rules. Both show each reply as the reason of a blocked prompt, and in nookku view. docs/how-to/choose-a-relay.md compares them. Then read the guide of your setup: Claude Code plugin, Claude Code project hooks or Codex.

Results

Data: plugin in Claude Code, plugin in Codex, project hooks in Claude Code, project hooks in Codex, evaluation. Method: docs/results.md.

Under pressure, the mechanism had 0 breaks in 1,000 turns. The test had 40 scripted conversations in each harness, 5 or 20 turns long. They had refusals, HTTP 500 errors, questions back to the tester and ambiguous messages. Each turn was a new harness call. I registered the design before the first run. Method, data and the one deviation: docs/results.md.

Docs

The how-to rows are in the order of the tasks of a test: choose, connect, run, read the results, fix. The row after them has the tasks that maintain Nookku.

The full map is docs/index.md. Read docs/limits.md before you trust a result: the deny is best effort, and the entry is not audited.

Contribute

Read CONTRIBUTING.md and the code of conduct. Report a security problem as SECURITY.md says.

License

Apache-2.0. See LICENSE.