xorcise 0.1.0

XORCISE — evaluate cyber AI agents.

Quickstart · Output · Agents · Missions · Open source · Website · Docs · Contributing · Security

---

Run your cyber-AI agent against a real mission. Watch everything it does. Grade the evidence.

AI can take action. It cannot bear consequences.

A benchmark score tells you an agent finished. It says nothing about the destructive commands it tried on the way there. XORCISE runs the agent against a live target inside a contained environment, records every command, tool call and dead end as OpenTelemetry evidence, and grades that evidence against the mission's own criteria.

Trust is not declared. It is demonstrated.

Quickstart

Tested on Ubuntu. Needs Python 3.12+ and Docker Engine.

pip install xorcise
xorcise up                                # boots the stack, prints the console URL

xorcise agent register --name my-agent
xorcise mission list
xorcise run create --agent my-agent --mission demo
xorcise run prompt <run_id>               # the ready-to-paste connect prompt
xorcise run status <run_id>               # score, breakdown, evidence

xorcise doctor checks the host first. xorcise down stops it all. No Docker on the box? xorcise up --stub is the self-contained demo. xorcise --help has the rest, and docs.xorcise.ai walks through a first run end to end.

Prefer to work from source? See Contributing → Setup.

What comes out

Live trace	every command, tool call and message, streaming into the console as it happens
Score	deterministic checks plus a bring-your-own-model judge
Report	the full run record, exportable — Markdown, HTML, JSONL
Leaderboard	agents ranked across recorded results

Every run gets its own private network and a fresh environment, created for the run and destroyed after it. An agent under evaluation cannot reach the host, or another run.

Bring your agent

XORCISE evaluates the agent you already use.

OpenHands	full trace + tool-call capture
Claude Code	via OTLP telemetry
Codex CLI	via OTLP telemetry
Anything custom	register it, drive it with the connect prompt, submit over REST

Activity is normalised into one event model, so the trace, the grading and the report read the same whichever harness produced the run.

Missions

A mission is a self-contained target: services, a network, and the criteria an agent is graded against. Packaged as bundles, pulled on demand.

Missions are deliberately vulnerable — SQL injection, IDOR, network pivots. That is the point: they exist so an agent has something real to find.

Run XORCISE on infrastructure you are willing to lose — a dedicated VM or an isolated cloud environment, never a workstation holding credentials you care about. It executes untrusted agent code against vulnerable targets by design.

Open source

XORCISE goes public in parts, not whole. This repository is the engine — the CLI, harness adapters, isolation, grading and console — under Apache-2.0, with issues and pull requests open.

The evaluation technology is open source. The commercial layer — managed deployment, runtime, command and sovereign hosting — is not. The agent skills and the documentation source are published separately as they are readied.

Documentation & help

xorcise.ai	the project website — what XORCISE is and who it is for
Documentation	first run, missions, grading, traces, the full CLI and API reference
Contributing	dev setup, the test lanes, the PR process, versioning
Security	what's in scope, and how to report privately
Maintainers · Code of Conduct	who to ask, and how we work

Found a vulnerability? Do not open a public issue — report it privately. Flaws in the harness, the isolation boundary or the supply chain are in scope; flaws inside a mission are the content.

License

Apache-2.0 © The XORCISE Authors

---

XORCISE.AI — Trust Evidence, not Claims.