trap
Non-invasive CLI testing framework for AI prompts, agents, and workflows.
trap treats any solution as a black box — it invokes it as a subprocess, captures outputs, then optionally scores them through a language-agnostic judge and grader. The solution doesn't need to import trap or know it exists.
Install
# requires uv — https://docs.astral.sh/uv/getting-started/installation/
# from PyPI
uv tool install trap-cli
# from git (latest main)
uv tool install "git+https://github.com/trapstreet/trap.git"
The command is tp.
How it works
trap runs your solution as a subprocess with one env var (TRAP_MANIFEST — an input dir and
an output dir), captures what it writes, then optionally scores it through a judge (per case)
and a grader (overall). See the full flow and IO contract.
Two roles, two directories, one IO contract:
- Solution author — writes
trap.yamland the solution code - Task author — writes
traptask.yaml,inputs/,expected/, and optional judge/grader scripts
Quick start
# from examples/echo/solution/
tp run # run all cases
tp run -t smoke # run only cases tagged `smoke`
Documentation
License
MIT
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file trap_cli-0.0.14.tar.gz.
File metadata
- Download URL: trap_cli-0.0.14.tar.gz
- Upload date:
- Size: 52.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: uv/0.12.2 {"installer":{"name":"uv","version":"0.12.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a3f778d3d2ef56e10ed7bd70d0e791c6c3b7aab0811ee091095559bf0b75b2cd
|
|
| MD5 |
74ac5997c1a823f71ed76a72aacf763e
|
|
| BLAKE2b-256 |
58619a98f7042a49a0a9570ccf4912b53f3ef46f38243c2b278bcadf047f6c55
|
File details
Details for the file trap_cli-0.0.14-py3-none-any.whl.
File metadata
- Download URL: trap_cli-0.0.14-py3-none-any.whl
- Upload date:
- Size: 71.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: uv/0.12.2 {"installer":{"name":"uv","version":"0.12.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2b296cdc16f9e90abf103b77649341e851ca7dd84278a773ec83e3dfd78d1c50
|
|
| MD5 |
4befdecf6fd0c95242cd6b43a715a318
|
|
| BLAKE2b-256 |
93a85cbf9005a2298d966197a6a79eac32e2642d9022951708274d95d20ee887
|
