agent-ultra-kit

Adversarial panel · ULTRA loop · command broker · proof gates for AI agent runtimes.

Tests prove the known contract. An adversarial panel finds the unknown failure modes. Proof gates decide what ships. A command broker makes host execution powerful and accountable. This kit packages those patterns as a portable Python 3.10+ core (stdlib only) with optional adapters.

Release status

Version 0.2.1 is the current release. It includes the original panel, ULTRA loop, command broker, proof/evidence/artifact layers, routes, memory hooks, adapters, and install CLI, plus ultracode, bob, the receipts bus, verifier, structural panel receipts, leak gate, and the optional worker layer. See CHANGELOG.md for the full release notes.

How to give this to your AI agent

The fastest way in: paste this into Claude Code, Cursor, aider, or your own agent, and it installs, configures, and proves the kit for you.

Install agent-ultra-kit 0.2.1 into this project and prove it works. Install with python -m pip install agent-ultra-kit==0.2.1, run agent-ultra init, then agent-ultra doctor and agent-ultra demo and show me the output — the demo must end with "DEMO PASSED". Then run agent-ultra panel "What could make this module unsafe to ship?" --evidence-dir ./src (add --mock if I have no model endpoint configured). If anything fails, read docs/troubleshooting.md in the repo, fix it, and rerun.

The full handoff prompt (with model-route config and rollback) is in INSTALL.md.

Install it yourself

PyPI:

python -m pip install agent-ultra-kit==0.2.1

Immutable Git tag:

python -m pip install "git+https://github.com/trollbot2012/agent-ultra-kit.git@v0.2.1"

The bootstrap scripts install the immutable GitHub archive for tag v0.2.1, verify that version, run doctor in a staged environment, and retain the previous environment for rollback. Download and inspect the script before running it; see INSTALL.md.

See it work (no API key)

The deterministic mock route exercises the CLI without a model endpoint or API key:

agent-ultra doctor
agent-ultra demo                    # success ends with: DEMO PASSED
agent-ultra --mock panel "Is this auth service safe?" \
  --lenses security,correctness,failure-modes

Then point it at a compatible /chat/completions endpoint:

agent-ultra init                      # writes agent-ultra.yaml + .env.example
# edit agent-ultra.yaml: base_url, routes; put your key in the env var it names
agent-ultra doctor --live             # probe the endpoint
agent-ultra panel "What could make this change unsafe to ship?" --evidence-dir ./src

ultracode — deterministic multi-agent workflows

Where the panel debates one question, ultracode runs a script that fans work across many bounded agents and proves what happened. A workflow is a plain Python module — META + async def run(wf) — using wf.agent (one model call, optionally schema-validated), wf.parallel (barrier), wf.pipeline (no barrier), wf.budget (hard call/token ceilings), and wf.run_check (broker-gated host commands). Every run writes a replayable journal and a checksummed receipt; a terminal status card renders from the journal, so a model's own text can never fake progress.

agent-ultra ultracode run smoke --mock
# Success prints "Status: COMPLETE" and a concrete resume command.
# Run that printed command to verify journal replay.

Commands: ultracode run <workflow> · list · status · resume <run_id>. Bundled workflows: smoke (fan-out + pipeline) and review (finders → skeptic votes → synthesis). Full guide: docs/ultracode.md.

bob — the 10-step enforced build pipeline

bob composes the whole kit into one enforced build loop (alias: agent-ultra build):

SPEC -> RED -> GREEN -> REFACTOR -> CODE-QUALITY -> SECURITY-FANOUT
     -> WORKFLOW -> ULTRA -> QUIZ -> COMMIT

Every gated step leaves a hash-chained, HMAC-signed receipt written from real execution — the pytest runner's actual output (RED/GREEN), ultracode's checksummed run receipts (the two fan-out steps), the panel's execution receipt (ULTRA). The commit gate re-derives what it can (a live pytest re-run, file staleness hashes) and cross-checks the rest. You cannot claim a step ran:

• a skipped step blocks (missing receipt + broken chain),
• a fabricated fan-out blocks (no ultracode run receipt backs the claim),
• a doctored panel receipt blocks (its checksum no longer matches),
• an edited or hand-authored receipt blocks (integrity/HMAC fail).

python -m pip install pytest
agent-ultra bob run "add a slugify helper" --mock
# Success prints "gate PASSED" after the offline pipeline completes.

Mock mode swaps only the model content for a bundled sample task — pytest, ultracode, the panel, and the gate all really execute, which is what makes the no-key demo honest. bob's one runtime dependency is pytest (python -m pip install pytest, or python -m pip install ".[bob]"); the rest of the kit needs nothing.

Commands: bob run <task> · bob gate --mark-pass (pre-commit: validates, keeps the run active) · bob seal (post-commit: releases it) · bob hook-install (writes all three hooks) · bob status. Full guide: docs/bob-the-builder.md.

What's in the box

Adapters (all optional): generic CLI, LiteLLM, Docker sandbox, external memory, Hermes-style and Ktisis-style runtimes.

Workers: router (default) vs Deep Agents

Ultra is the supervisor and proof gate — it decides what ships. A worker only fills the builder/fixer slots:

• Router worker (default) — stdlib, one model call per fix, returns an advisory single-file edit that the loop applies with automatic rollback. Cheap, portable, Windows-native, zero dependencies.
• Deep Agents worker (optional) — a multi-step LangChain Deep Agents runtime for from-scratch multi-file builds and large repairs. Installed only via the extra on Python 3.11+; imported only when selected. The stdlib core and router worker remain supported on Python 3.10.

python -m pip install agent-ultra-kit==0.2.1              # router worker
python -m pip install "agent-ultra-kit[deepagents]==0.2.1" # optional worker

agent-ultra ultra "fix the finding" --workspace .                    # router (default)
agent-ultra ultra "build a service" --workspace . --build --worker deepagents

Both workers return the same WorkerResult shape, so Ultra never cares which one produced a change — the edit still passes tests, the panel, and the proof gate before it can ship. Deep Agents is a worker, not the ship authority.

Panel execution receipts

Tests prove known contracts.
Panels find unknown failure modes.
Panel execution receipts prove the panel actually RAN.
Proof gates decide what ships.

A phase labelled PANEL is not proof. When the loop runs a panel it writes panel_execution_receipt.json into the run dir, built from the REAL PanelReport (model_calls, lenses, per-finding origins) with a mandatory integrity checksum. Before REPORT the loop validates it (UltraReport. panel_enforced), and agent_ultra.panel_receipt.gate_report(run_dir) (also agent-ultra panel-gate <run_dir>) blocks REPORT unless the receipt shows real executed lenses. A self-review produces no receipt with lens_count_executed > 0, so it cannot pass:

PANEL phase completed with 0 agent calls — self-review is not a panel.
REPORT blocked: missing or invalid panel execution receipt.

Stdlib, additive — it does not weaken the existing proof gates.

Use it with YOUR agent

Your agent needs exactly one thing — a way to call a model:

from agent_ultra import PanelEngine, RoutePool, OpenAIChatClient

pool = RoutePool(["your-model"],
                 client=OpenAIChatClient("https://your-endpoint/v1",
                                         api_key_env="YOUR_KEY_ENV"))
report = PanelEngine(pool).run("Is this safe to ship?", evidence_dirs=["./src"])
if report.accepted:
    ...  # feed report.accepted into your agent's task queue

Wire the deeper loop with two callables that drive your coding agent:

from agent_ultra import UltraLoop, CommandBroker, TRUSTED_OWNER_TIERS

loop = UltraLoop(".", panel=engine,
                 broker=CommandBroker(ledger_path=".ultra/broker.jsonl",
                                      auto_run_tiers=TRUSTED_OWNER_TIERS),
                 builder=my_agent_builds,     # (workspace, task) -> str
                 fixer=my_agent_fixes)        # (fix_task, workspace) -> bool
report = loop.run("add token auth", risk="high", test_cmd="pytest -q")
print(report.shipped, report.ship_reason)

Full guide: docs/adapter-guide.md.

Docs

architecture · panel · ULTRA loop · ultracode · bob pipeline · command broker · proof gates · adapter guide · receipts bus · verifier · security · troubleshooting · releasing · INSTALL · CHANGELOG

Security posture (short version)

Model output is untrusted input. Dangerous commands deny without an approval path. Critic-proposed checks auto-run only if they are pure reads. Secrets are redacted from evidence, ledgers, and artifacts. Keys live in env vars the config only names. Details: docs/security.md.

Development

git clone https://github.com/trollbot2012/agent-ultra-kit.git
cd agent-ultra-kit
python -m pip install -e ".[dev]"
python -m pytest -q
python examples/basic_panel/run.py
python examples/command_broker_demo/run.py
python examples/ultra_loop_demo/run.py

MIT license.