runopsy-bench
Labelled traces with known ground truth, and the metrics that score a diagnosis against them.
This package is the difference between claiming the engine localizes failures and knowing it does. Every case declares where the run actually broke, so onset accuracy, top-3 recall and step distance are measured rather than asserted.
Two design choices keep the number honest.
Negative controls are part of the suite. A healthy run that produces no finding is scored alongside the failures. An engine that flags everything would otherwise post a perfect onset accuracy while being useless, and false positives are what actually get a diagnosis tool switched off.
Known blind spots are labelled, not hidden. Some cases — a tool that succeeds while
writing the wrong value, an agent that picks a plausible but wrong tool — cannot be
found by structural analysis at all, because nothing in the recorded trace is anomalous.
They are marked deterministically_detectable = False, excluded from the accuracy
figure, and reported separately as the coverage gap that semantic analysis exists to
close. Quietly dropping them would inflate the headline number and hide exactly the work
that remains.
1 maintainer
Metadata
1 maintainer
Release files for runopsy-bench 0.1.9
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| runopsy_bench-0.1.9.tar.gz | 26.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| runopsy_bench-0.1.9-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 60.9 kB
Release files / runopsy_bench-0.1.9.tar.gz
| Download URL | runopsy_bench-0.1.9.tar.gz |
|---|---|
| Size | 26.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
07ffb4bb0b7afdc1ae25a29f5acb343cb2ad8570a7b05e8c52a82915fd2c1fd1
|
|
BLAKE2b-256 checksum How to use checksums |
891e84dceb86ee2b0a06d7903d7588fe755d343da33e7a026626185f28ab4ff8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.0 {"installer":{"name":"uv","version":"0.12.0","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / runopsy_bench-0.1.9-py3-none-any.whl
| Download URL | runopsy_bench-0.1.9-py3-none-any.whl |
|---|---|
| Size | 34.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
fc84a38ce9b6458cebccba2ab6f1680c53d8a8ae6c79954e4b27d6afafef5c3e
|
|
BLAKE2b-256 checksum How to use checksums |
9e86a5d525afa1deede5f232503fa3dc44d44df7520bc3bec88e1ad0e8ffe881
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.0 {"installer":{"name":"uv","version":"0.12.0","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
