This release is a pre-release and may not be stable for production use.
Verdict Eval
PyPI distribution: cognifity-verdict-eval. Python import: verdict_eval.
The Verdict eval engine. LLM-as-judge with binary rubric, intent clustering, non-parametric drift detection per cluster per dimension (Fisher's exact test for binary PASS/FAIL dimensions, Mann-Whitney U for continuous metrics), Bradley-Terry pairwise comparator for cross-LLM evaluation, and a synthetic regression injector for verifying the pipeline catches what it should.
The repository pipeline uses a persisted cluster registry rather than
re-clustering the full dataset on every run. Local
sentence-transformers/all-MiniLM-L6-v2 embeddings are the semantic default;
0.50 cosine distance is the shipped starting threshold, not a universal
cutoff. The built-in hash embedder is an explicit lexical fallback. Change the
embedding model or threshold only with a new clustering version and a one-time
--recluster for existing traces. --trust-existing-clusters is reserved for
stable cluster IDs assigned outside Verdict.
Drift is a batch comparison over each captured trace's started_at time. The
runner defaults to a 24-hour current window and a 7-day baseline separated by a
24-hour gap, with at least 30 judgments per (cluster, dimension) window. A
signal must clear the BH-adjusted p-value gate and the Cliff's delta effect-size
gate. On binary PASS/FAIL data the default 0.147 delta is a 14.7 percentage-
point sensitivity floor. These defaults require workload-specific validation.
Pipeline reruns deduplicate by trace for the selected judge model and rubric version. Stored judgments from a different evaluator definition are retained but are not pooled into the current drift windows.
from verdict_eval import Judge, DEFAULT_RUBRIC, DriftDetector, CorruptionInjector
See the repository README, ADR-002, and the verification scripts.
Apache 2.0.
1 maintainer
Release files for cognifity-verdict-eval 0.1.0a3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| cognifity_verdict_eval-0.1.0a3.tar.gz | 86.6 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| cognifity_verdict_eval-0.1.0a3-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 158.3 kB
Release files / cognifity_verdict_eval-0.1.0a3.tar.gz
| Download URL | cognifity_verdict_eval-0.1.0a3.tar.gz |
|---|---|
| Size | 86.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f1e4f472081eb5faa5552f5b1c24af011c59b7bcc6cd85cd6513cd1eead0e913
|
|
BLAKE2b-256 checksum How to use checksums |
6cdca0318d27d5bfccce4712e0c9d7de94727e122cde6f9e37e32265e74527b5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 11, 2026.
Transparency logRelease files / cognifity_verdict_eval-0.1.0a3-py3-none-any.whl
| Download URL | cognifity_verdict_eval-0.1.0a3-py3-none-any.whl |
|---|---|
| Size | 71.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
18075ad4543f0eaf40a4cdaaf934ac6fc327647ace300c59f737d92622c87c7e
|
|
BLAKE2b-256 checksum How to use checksums |
6e0fb3ce530cee966c3e88f989d285ff0e6e513a3b2c5940c7dce5449e04f655
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 11, 2026.
Transparency log
