trap
Non-invasive CLI testing framework for AI prompts, agents, and workflows.
trap treats any solution as a black box — it invokes it as a subprocess, captures outputs, then optionally scores them through a language-agnostic judge and grader. The solution doesn't need to import trap or know it exists.
Install
# requires uv — https://docs.astral.sh/uv/getting-started/installation/
# from PyPI
uv tool install trap-cli
# from git (latest main)
uv tool install "git+https://github.com/trapstreet/trap.git"
The command is tp.
How it works
trap runs your solution as a subprocess with one env var (TRAP_MANIFEST — an input dir and
an output dir), captures what it writes, then optionally scores it through a judge (per case)
and a grader (overall). See the full flow and IO contract.
Two roles, two directories, one IO contract:
- Solution author — writes
trap.yamland the solution code - Task author — writes
traptask.yaml,inputs/,expected/, and optional judge/grader scripts
Quick start
# from examples/echo/solution/
tp run # run all cases
tp run -t smoke # run only cases tagged `smoke`
Documentation
License
MIT
1 maintainer
Metadata
1 maintainer
Release files for trap-cli 0.0.14
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| trap_cli-0.0.14.tar.gz | 52.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| trap_cli-0.0.14-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 124.2 kB
Release files / trap_cli-0.0.14.tar.gz
| Download URL | trap_cli-0.0.14.tar.gz |
|---|---|
| Size | 52.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a3f778d3d2ef56e10ed7bd70d0e791c6c3b7aab0811ee091095559bf0b75b2cd
|
|
BLAKE2b-256 checksum How to use checksums |
58619a98f7042a49a0a9570ccf4912b53f3ef46f38243c2b278bcadf047f6c55
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.2 {"installer":{"name":"uv","version":"0.12.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / trap_cli-0.0.14-py3-none-any.whl
| Download URL | trap_cli-0.0.14-py3-none-any.whl |
|---|---|
| Size | 71.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
2b296cdc16f9e90abf103b77649341e851ca7dd84278a773ec83e3dfd78d1c50
|
|
BLAKE2b-256 checksum How to use checksums |
93a85cbf9005a2298d966197a6a79eac32e2642d9022951708274d95d20ee887
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.2 {"installer":{"name":"uv","version":"0.12.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
