Skip to main content

evalX

pytest for AI behavior. Grade LLM apps with golden datasets, code checks + an LLM judge, track every run locally, and gate CI on every single case.

CI Python 3.11+ License: MIT Tests: 296 passing

Install · Quickstart · Features · Check types · CLI reference · Dashboard · Milestones

Run detail for the AeroIntel dogfood run: 67 cases, mean 0.737, judged by claude-haiku-5-5

evalX grading a real system: the AeroIntel dogfood run — 67 cases, mean 0.737.

Local-first by design. Every run lives in a single DuckDB file inside your repo — no account, no cloud, no telemetry. Your eval history sits next to your code, and the whole loop works offline with a local judge model. promptfoo is a platform; evalX is the 10-minute, local-first regression catcher for Python LLM apps.

Features

  • Golden datasets in YAML — hand-written cases with weighted code checks (exact_match, contains, not_contains, regex, numeric, max_length, json_schema) and optional LLM-judge rubrics.
  • LLM-as-judge with bias guards — answers scored alone (no position bias), an anti-fluff system prompt (no verbosity bias), verdicts validated as Pydantic JSON (retry once, then raise — never an invented score).
  • Any text-in → text-out target — a Python function via module:attr, a hosted service via a small adapter, or any OpenAI-compatible endpoint (Ollama, vLLM, LM Studio, LiteLLM proxies, …) via the generic adapter.
  • Local run store — every run saved to one DuckDB file in your repo; compare any two runs case-by-case.
  • Quality vs cost — judge tokens and dollar cost are tracked per case and per run (priced by LiteLLM, retries included); evalx compare shows latency and cost changes next to the score deltas.
  • Per-case regression gate — pin a trusted run as a baseline; CI fails if any single case drops past your threshold, even when the mean improves. A guard refuses baselines from a different dataset.
  • Errors can't hide — a crashing case is recorded with its error and scored 0.0, runs carry error/skipped counts, an infra gate (--max-error-rate) fails CI on outage fraction before the score gates, and a run with errors is refused as a baseline.
  • Streamlit dashboard — run history with trends, per-case drill-down (question, answer, per-check scores, judge reasoning), and run compare.
  • CI-ready — exit-code gates for mean score, per-case regression, and error rate; the included GitHub Actions workflow runs lint, tests, and an eval gate on every push.

Proof it works

evalX has graded two real systems (plus its own deterministic demo):

Target Suite Result
AeroIntel — local aviation-maintenance RAG 67 cases (35 answerable, 32 must-refuse), judged by Claude Haiku 5.5 mean 0.737; refusals 31/32; one case rescued by the judge (code 0.00 → judge 1.00)
Ollama qwen3:4b-instruct (local) 6 general cases via the generic adapter mean 0.833, total cost $0
Demo app (deterministic, deliberately mediocre) 6 cases 0.500 by design — the CI gate's calibration fixture

296 tests (including end-to-end), ruff clean, CI green on every push.

Install

pip install evalx-gate

The package is evalx-gate on PyPI (an unrelated project owns the evalx name there); the command is evalx and the import is import evalx. Point --target at a module:function in your project, run from your project's folder. Python 3.11+.

Quickstart

From a clone of this repo (the demo dataset and app live here):

pip install -e .
evalx run --suite datasets/demo/v1 --target examples.demo_app:answer

The demo target is deliberately naive: it aces easy questions, gets partial credit on medium ones, and hallucinates on adversarial cases. Watch the harness catch all of it — mean score 0.500 over 6 cases.

Plug in your own app

evalX calls one function in your app, answer(question: str) -> str, named with --target module:function. Everything inside it (retrieval, prompts, the model) stays your app's business. The tiny RAG example walks through the whole loop offline in a minute: grade a toy RAG app, pin a baseline, switch its "model" and watch one change improve a case and another fail the gate.

evalx run --suite examples/tiny_rag/cases.yaml --target examples.tiny_rag.app:answer
evalx baseline pin latest --name tiny-rag
TINY_RAG_MODEL=v3 evalx run --suite examples/tiny_rag/cases.yaml \
  --target examples.tiny_rag.app:answer --baseline tiny-rag
# REGRESSED refuse-offtopic: 1.00 -> 0.00 (drop 1.00) -> exit 1

Gate a run in CI:

evalx run --suite datasets/demo/v1 --target examples.demo_app:answer --fail-under 0.8
# exits 1: mean 0.500 < gate 0.8

Speed up a big suite by running cases concurrently:

evalx run --suite datasets/aerointel/v1 --target examples.aerointel_app:answer --workers 8

Results keep dataset order and score exactly as a sequential run would. Keep --workers within your target's rate limit (the AeroIntel adapter retries HTTP 429 with Retry-After).

LLM answers vary between identical calls. Measure how much with --repeats:

evalx run --suite datasets/aerointel/v1 --target examples.aerointel_app:answer \
  --repeats 3 --workers 4

Each case is asked 3 times; its score is the mean and a spread column (max − min across attempts) shows how noisy it is. Cases that move more than the regression threshold between identical attempts are listed as noisy, and a REGRESSED line on such a case is marked (noisy: … may be model noise) — the gate still fails, but you can see whether to trust it. The stored answer and checks are from the lowest-scoring attempt, the one worth debugging.

Measuring noise: where to set the bar

The regression bar (--fail-on-regression, default 0.2) should sit just above how much your app moves on its own. evalx noise reads a saved --repeats run and works it out:

evalx run --suite datasets/aerointel/v1 --target examples.aerointel_app:answer --repeats 3
evalx noise latest
6 of 67 cases moved at all.
Normal wobble (largest spread after setting aside the noisiest 3 cases): 0.07
Suggested bar: 0.20   → --fail-on-regression 0.2

Noisy (move more than the bar between identical attempts):
  user-bolts-nuts-safetying   0.33   attempts 0.17, 0.50, 0.50
  user-fabric-inspect-repair  0.33   attempts 0.17, 0.17, 0.50

Normal wobble is the largest spread left after setting aside the noisiest 5% of cases (at least one), so a couple of outliers can't define "normal" while ordinary wobble, like small judge variation, stays inside the bar. The suggested bar is 2.5× that, rounded up to 0.05, between 0.05 and 1.0 (--trim and --multiplier change the rule). Noisy cases move more than the bar: fix them in the app rather than raising the bar for everyone. It needs a run with at least 2 attempts per case (3 or more is better). Measure again whenever the app's model or the judge changes.

Rejudge: is it the app or the judge?

A score moves when the app answers differently or when the judge grades the same answer differently. evalx rejudge separates the two: it re-grades a run's stored answers with the judge only, several times, without calling the app.

evalx rejudge latest --repeats 3                    # how consistent is my judge?
evalx rejudge latest --judge-model anthropic/claude-sonnet-5-5   # would another judge agree?
evalx noise latest                                  # judge-only wobble of the re-grade
evalx compare <original> latest                     # score changes caused by the judge alone

It reuses each case's stored answer and code-check results, takes the rubric from the dataset (--suite if it moved), and saves the re-grade as a new run linked to the original. Cases without a rubric are carried over unchanged; cases that errored or are gone from the dataset are skipped and listed. On a --repeats run it re-grades the one stored answer (the lowest-scoring attempt), so compare the spreads rather than the two judge columns. It costs judge calls only.

How it works

One command runs the whole pipeline — load the exam, ask your app, grade every answer, remember the run:

flowchart LR
    subgraph S1["📥 PHASE 1 · SETUP"]
        direction TB
        CLI(["⚡ evalx run — you type the command"])
        PARSE["cli.py — flags → arguments; judge built only if asked"]
        LOAD["datasets.py — YAML → TestCases, Pydantic-validated"]
        RESOLVE["adapters.py — module:attr → real function"]
        CLI --> PARSE
        PARSE --> LOAD
        PARSE --> RESOLVE
    end
    subgraph S2["🔁 PHASE 2 · THE LOOP — repeats per case"]
        direction TB
        ASK["Ask the app — answer = answer_fn(question), timed"]
        CODE["Code checks — scorers.py: exact · contains · regex"]
        RUBRIC{"rubric?"}
        LLM["LLM judge — judge.py → LiteLLM, JSON verdict validated"]
        SKIP["judge skipped"]
        BLEND["Blend by weight — _combine_score"]
        CASE["CaseResult — score · latency · reason"]
        ASK --> CODE
        ASK --> RUBRIC
        RUBRIC -->|yes| LLM
        RUBRIC -->|no| SKIP
        CODE --> BLEND
        LLM --> BLEND
        SKIP --> BLEND
        BLEND --> CASE
        CASE -.->|"↺ next case"| ASK
    end
    subgraph S3["📤 PHASE 3 · REPORT + MEMORY"]
        direction TB
        MEAN["RunResult — mean of all case scores"]
        SAVE[("store.py — DuckDB .evalx/runs.duckdb")]
        TABLE["Report — Rich table; gate → exit 0 / 1"]
        CMP["evalx compare — latest-1 vs latest, per-case deltas"]
        MEAN --> SAVE
        MEAN --> TABLE
        SAVE -.-> CMP
    end
    LOAD --> ASK
    RESOLVE --> ASK
    CASE --> MEAN

    classDef input fill:#16283f,stroke:#3b82f6,stroke-width:2px,color:#e6e9f0;
    classDef exec fill:#14301f,stroke:#22c55e,stroke-width:2px,color:#e6e9f0;
    classDef score fill:#2a2145,stroke:#a78bfa,stroke-width:2px,color:#e6e9f0;
    classDef judge fill:#3a2a10,stroke:#f59e0b,stroke-width:2px,color:#e6e9f0;
    classDef out fill:#12303d,stroke:#38bdf8,stroke-width:2px,color:#e6e9f0;
    class CLI,PARSE,LOAD,RESOLVE input;
    class ASK,CASE exec;
    class CODE,BLEND,SKIP score;
    class RUBRIC,LLM judge;
    class MEAN,SAVE,TABLE,CMP out;

Check types

Each case lists weighted checks; a case's code score is their weighted mean. Checks are validated when the dataset loads, so a typo fails fast instead of silently scoring 0.

Type Passes when expected
exact_match the answer equals the text (surrounding whitespace ignored) text
contains the text appears in the answer (case-insensitive) text
not_contains the text does not appear — a banned phrase, a hallucinated name, a leaked secret text
regex the pattern matches anywhere in the answer Python regex
numeric any number in the answer is within tolerance (default 0) of the value; reads 1,200, -3, .5, 40psi number
max_length the answer has at most this many characters positive integer
json_schema the answer is JSON that validates against the schema (one Markdown code fence allowed) JSON Schema, as a YAML mapping
- id: torque-json
  question: "Give the torque for an AN3 bolt as JSON."
  checks:
    - type: json_schema
      expected:
        type: object
        required: [part, torque_in_lb]
        properties: {torque_in_lb: {type: number}}
    - {type: numeric, expected: 25, tolerance: 3}
    - {type: not_contains, expected: "ft-lb"}   # wrong unit
    - {type: max_length, expected: 200, weight: 0.5}

CLI reference

Command What it does Key options
evalx run Run a suite against a target, print the per-case table, save the run --suite, --target, --judge-model, --judge-temperature, --fail-under, --baseline, --fail-on-regression (default 0.2), --max-error-rate (default 0.0), --workers (default 1), --repeats (default 1), --report FORMAT:PATH (repeatable), --no-save
evalx runs List stored runs, newest first, with mean, error/skipped counts, judge cost, wall time and workers —
evalx compare A B Per-case score, latency and judge-cost deltas plus whole-run wall time between two runs (refs: full id, unique prefix, latest, latest-N) —
evalx baseline pin REF Pin a stored run as a named baseline (--name, default stable) --allow-errors (required if the run contains error cases)
evalx baseline list List pinned baselines —
evalx dashboard Launch the Streamlit dashboard --port (default 8501)
evalx noise [RUN] Wobble between identical attempts of a --repeats run, and a suggested --fail-on-regression bar --trim (default 0.05), --multiplier (default 2.5)
evalx rejudge RUN Re-grade a run's stored answers with the judge only (the app isn't called) --repeats (default 3), --judge-model, --suite, --workers, --no-save
evalx version Print the version —

What's here

  • evalx/schemas.py — Pydantic models: TestCase, Check, JudgeSpec, CaseResult, RunResult
  • evalx/datasets.py — load + validate golden datasets from YAML
  • evalx/scorers.py — deterministic scorers: exact_match, contains, not_contains, regex, numeric, max_length, json_schema
  • evalx/judge.py — LLM-as-judge via LiteLLM with built-in bias guards: answers scored alone (no position bias), anti-fluff system prompt (no verbosity bias), JSON verdicts validated with Pydantic (retry once, then raise instead of inventing a score), temperature optional — some models (e.g. claude-haiku-5-5) reject a forced temperature, so the parameter is omitted unless you set it
  • evalx/adapters.py — resolve module:attr targets to question → answer callables
  • evalx/runner.py — run every case, combine code checks + judge by weight; a failing case is recorded with its error, never aborts the run
  • evalx/store.py — DuckDB run store: save runs, list them, resolve latest / latest-N / id-prefix references, compare two runs per case, pin and check baselines
  • evalx/cli.py — the Typer CLI above (Rich tables, exit-code gates)
  • evalx/dashboard.py — Streamlit dashboard: run history, per-run detail, run-over-run compare
  • .github/workflows/ci.yml — CI on every push: lint, tests, and the demo suite gated at --fail-under 0.4
  • datasets/demo/v1/cases.yaml — 6 hand-written golden cases (2 with rubrics)
  • datasets/aerointel/v1/cases.yaml — 67 cases converted from AeroIntel's golden set
  • datasets/general/v1/cases.yaml — 6 general-knowledge cases for bare LLMs
  • examples/demo_app.py — deliberately mediocre demo target
  • examples/tiny_rag/ — a toy RAG app with golden cases and three "model" versions: the quickest way to see how a target plugs in
  • examples/aerointel_app.py — HTTP adapter for AeroIntel (token auth, 429 retry with Retry-After)
  • examples/openai_compatible_app.py — generic adapter for any OpenAI-compatible endpoint (Ollama by default)

Judge usage

export ANTHROPIC_API_KEY=...   # or the key for your judge provider
evalx run --suite datasets/demo/v1 --target examples.demo_app:answer \
  --judge-model anthropic/claude-haiku-5-5

Only cases with a judge: rubric invoke the judge; the rest use code checks. Two judged cases on Haiku cost a fraction of a cent. Cases with a rubric and no checks are skipped when you run without --judge-model (listed in the output, left out of the mean) rather than scored 0.

A case never aborts the run: if your target or the judge raises, that case records the error (shown in the CLI and the dashboard drill-down) and scores 0.0, so a crashing app can't slip past the gate.

Run store & comparison

Every evalx run is saved to .evalx/runs.duckdb (override with EVALX_STORE):

evalx run --suite datasets/demo/v1 --target examples.demo_app:answer
# Run saved: 20261019-143022

evalx runs                 # list stored runs, newest first
evalx compare latest-1 latest   # per-case delta table + mean change

Run references accept a full id, a unique prefix, latest, or latest-N. --no-save skips storing a throwaway run. The runs list shows each run's error and skipped counts (– for runs stored before those were tracked).

Judged runs also record what the judge consumed: tokens and cost per case (retries included), totalled per run. evalx run prints the total, and evalx compare adds per-case latency and judge cost plus a summary, for example:

Mean: 0.500 -> 0.750 (+0.250) | 2 improved, 0 regressed, 4 unchanged
Latency (mean per case): 1200 -> 800 ms (-33%)
Judge cost: $0.0020 -> $0.0016 (-20%)
Wall time (whole run): 7.4s -> 5.1s (-31%) | workers 1 -> 1

Every run also records its wall-clock time and --workers. Compare prints them on a Wall time (whole run) line and notes when the two runs used different worker counts: per-case latency includes queueing at the target, so a parallel run can show higher latency per case and still finish sooner.

Cost comes from LiteLLM's price table. For a model it has no price for (a local or custom endpoint), tokens are still recorded and cost shows as –, never as a misleading $0.

Baselines & the regression gate (M5)

A mean-score gate can pass while one case craters. Pin a run you trust as a baseline, and every later run is also graded case-by-case against it:

evalx baseline pin 20261030-071230 --name aerointel   # bookmark a stored run
evalx baseline list                                   # see your bookmarks

evalx run --suite datasets/aerointel/v1 --target examples.aerointel_app:answer \
  --baseline aerointel                                # fail if any case drops > 0.2

A case fails the gate only when its score drops by more than the threshold (--fail-on-regression, default 0.2 — a drop of exactly 0.2 still passes). Cases added or removed since the baseline are reported as warnings, never failures: a changed dataset is not a regression.

A baseline is only meaningful against the same dataset — the comparison matches on case ids. If a run and its baseline share no cases at all, the gate refuses with an error instead of passing a comparison that never happened; if the baseline came from a differently-named suite, the shared cases are still compared, with a warning. Baselines live in the local run store, so a CI gate needs a store that contains the pin.

A run with error cases (the target or judge raised) is refused as a baseline unless you pass --allow-errors: a baseline built on an outage would hide every later regression. To keep an outage from passing as a quality drop in the first place, evalx run --max-error-rate (default 0.0) adds an infra gate that fails the run on its fraction of errored cases, reported before the score gates.

Reports for CI

evalx run --report FORMAT:PATH writes the run out for other tools, and can be given several times:

evalx run --suite datasets/demo/v1 --target examples.demo_app:answer --fail-under 0.4 \
  --report junit:reports/evalx.xml \
  --report markdown:"$GITHUB_STEP_SUMMARY" \
  --report json:reports/evalx.json
Format Use it for What's in it
junit test-report views (GitHub Actions reporters, GitLab, Jenkins) one <testcase> per case: <error> if it errored, <failure> if it regressed past the baseline or scored below --fail-under; score, judge reason, question and answer in system-out
markdown a job summary or a PR comment pass/fail per gate, judge cost, wall time, the cases needing attention (worst first) and a collapsible table of all cases
json anything else the full run plus every gate verdict, regressions and judge usage

Reports are written even when a gate fails, so CI can show why it failed. This repo's own CI writes the Markdown report to the workflow summary page.

Dashboard

evalx dashboard            # http://localhost:8501  (add --port N if busy)

Three tabs over your stored runs: Runs (history table with error counts and a mean-score trend line), Run detail (per-case scores for one run), and Compare (pick a baseline and a challenger — per-case deltas with ▲/▼ markers). Needs the dashboard extra: pip install "evalx-gate[dashboard]" (or pip install -e ".[dev]" in a clone).

Case drill-down: a must-refuse question, AeroIntel's refusal answer, and the regex check that graded it

Case drill-down — the stored question, the app's actual answer (here a refusal, returned verbatim by the adapter), and the per-check scores behind the final grade. The run-detail view of the same run is the screenshot at the top of this README.

Configuration

Variable Used by Purpose
EVALX_STORE run store Path of the DuckDB run-store file (default .evalx/runs.duckdb)
ANTHROPIC_API_KEY (or your provider's key) LLM judge API key for the --judge-model provider, via LiteLLM
AEROINTEL_URL AeroIntel adapter Base URL (default http://127.0.0.1:8000)
AEROINTEL_CLIENT_ID / AEROINTEL_CLIENT_SECRET AeroIntel adapter Token-auth credentials for AeroIntel's /auth/token
OPENAI_COMPAT_BASE_URL generic adapter Endpoint base URL (default http://localhost:11434/v1 — Ollama)
OPENAI_COMPAT_MODEL generic adapter Model name — required, no default
OPENAI_COMPAT_API_KEY generic adapter Optional bearer key for hosted endpoints
OPENAI_COMPAT_SYSTEM_PROMPT generic adapter Optional system prompt (handy for prompt A/B runs)
OPENAI_COMPAT_TEMPERATURE generic adapter Optional temperature; omitted unless set

Development

pip install -e ".[dev,e2e]"           # pytest, ruff, streamlit, pandas, playwright
python -m playwright install chromium # browser for the dashboard E2E test
python -m pytest                      # 296 tests
ruff check .

tests/e2e/ tests evalX end to end the way a user runs it: a real evalx process grades the OpenAI-compatible adapter against a local fake LLM server (real HTTP, real LiteLLM judge calls, real DuckDB store, real exit codes), and the dashboard is rendered in headless Chromium. No API key needed; the dashboard test skips if Playwright isn't installed. One opt-in check hits a real judge model: ANTHROPIC_API_KEY=... python -m pytest -m live.

CI (.github/workflows/ci.yml) runs the same lint + tests on every push and pull request, then gates the deterministic demo suite at --fail-under 0.4 (its designed mean is 0.500, so the gate verifies the grading itself).

Releasing

The version lives in one place, evalx/__init__.py. To release:

  1. Bump __version__ and add a ## [x.y.z] section to CHANGELOG.md (a test fails if the section is missing).
  2. Merge to main, then tag it: git tag vX.Y.Z && git push origin vX.Y.Z.

.github/workflows/release.yml then checks the tag matches the version, runs lint and tests, builds and twine checks the package, publishes it to PyPI through trusted publishing (no API token), and creates a GitHub release with that version's changelog notes.

Milestones

  • M1 — core harness: YAML golden datasets, code scorers (exact / contains / regex), runner, CLI with a mean-score gate.

  • M2 — LLM-as-judge: rubric scoring through LiteLLM with bias guards — Pydantic-validated JSON verdicts, retry once, never an invented score.

  • M3 — run store & compare: every run saved to a local DuckDB file; per-case deltas between any two runs.

  • M4 — dashboard + CI: Streamlit dashboard over the stored runs (history, case drill-down, compare); GitHub Actions runs lint, tests, and the eval gate on every push.

  • M5 — baselines & regression gate: pin a run and fail when any single case drops past a threshold, even if the mean improved — with a guard against baselines from a different dataset. See "Baselines & the regression gate" above.

  • Dogfooding: evalX graded AeroIntel, a local aviation-maintenance RAG, on a 67-case dataset built from its own golden set (35 answerable + 32 must-refuse): mean 0.737 with an LLM judge, with 31 of 32 must-refuse questions refused correctly. Dataset in datasets/aerointel/v1, adapter in examples/aerointel_app.py, screenshots in the Dashboard section above.

  • Concurrent runs: evalx run --workers N grades N cases at once on a thread pool, with results in dataset order — identical to a sequential run.

  • Quality vs cost: judge tokens and cost per case and per run, with latency and cost deltas next to score deltas in evalx compare and the dashboard.

  • Repeat runs: --repeats N scores every case N times — mean score, per-case spread, noisy cases flagged in the run and on regressions.

  • More check types: not_contains, numeric (with tolerance), max_length and json_schema, all validated at dataset load.

  • Reports for CI: --report junit|markdown|json:PATH, written even when a gate fails; CI posts the Markdown report to its summary page.

  • 0.2.0 on PyPI as evalx-gate, released from a version tag by GitHub Actions with trusted publishing. See the CHANGELOG.

  • 0.3.0 — noise tools: evalx noise suggests a regression bar from a --repeats run and lists noisy cases; evalx rejudge re-grades stored answers with the judge only, separating judge noise from app noise. On AeroIntel it showed the two noisy cases had zero judge wobble: the noise was the app. Plus examples/tiny_rag and a final RESULT: line on every run.

Roadmap

  • Every attempt's answer stored: today a --repeats run keeps the lowest-scoring attempt's answer; keeping all of them lets rejudge grade every attempt and compare cleanly against the source run.
  • 0.4 — semantic similarity and HTML export: a semantic check type scored with embeddings through LiteLLM (no torch), and evalx export --html for a self-contained report to share.
  • 0.5 — RAG faithfulness: targets can return their retrieved context; judge modes for faithfulness (is every claim supported?) and context relevance (did retrieval find the right chunk?).
  • 0.6 — multi-turn conversations: cases with several turns and checks per turn, for chatbots and agents.

License

MIT — see LICENSE.

Metadata

Release files for evalx-gate 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for evalx-gate 0.3.0
File Size Uploaded Provenance Actions
evalx_gate-0.3.0.tar.gz 105.9 kB With provenance Details for evalx_gate-0.3.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for evalx-gate 0.3.0
File Interpreter ABI Platform Actions
evalx_gate-0.3.0-py3-none-any.whl Python 3 none any Details for evalx_gate-0.3.0-py3-none-any.whl

Total release size: 157.0 kB

Release files / evalx_gate-0.3.0.tar.gz

Download URL evalx_gate-0.3.0.tar.gz
Size 105.9 kB
Tags Source
SHA-256 checksum
How to use checksums
50e60dde1f14dabb8c1a50d5908e2c7dec3760cca7b92a08aa99d1d8fe43bf5e
BLAKE2b-256 checksum
How to use checksums
9efc86edef93841f5de67ed9146382fdf453df8dc4c52451f7d4aab63ec2ddba
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 17, 2026.

Transparency log

Release files / evalx_gate-0.3.0-py3-none-any.whl

Download URL evalx_gate-0.3.0-py3-none-any.whl
Size 51.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
01d276afd96d23a63be70af50cae1f51ce0c06abd293357ffbd6bffd38e6dcde
BLAKE2b-256 checksum
How to use checksums
18c892b411fa8680ead1924d5e98cf07b180fbfc748e68b51eda2cfec61cc7f6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 17, 2026.

Transparency log

Security for evalx-gate 0.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for evalx_gate-0.3.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 17, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.3.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page