EvalSuite
Unified, reproducible evaluation for machine learning, LLM and research.
EvalSuite brings classification, regression, clinical, statistical, segmentation and object-detection evaluation into one consistent, validated, documented framework.
Status: stable (0.5.2). Every item on the 0.1.0–0.5.0 roadmaps is implemented and verified, including SPICE. LLM systems arrive in 0.5.0: safety, robustness, calibration and uncertainty, agents and tool use, multilingual, code generation, long context and summarization, and inference efficiency and cost.
Installation
pip install evalsuite-python
The package is installed as evalsuite-python and imported as evalsuite:
import evalsuite as es
Why EvalSuite
- One consistent API. Every metric returns a result object that behaves like a number and exports to JSON, pandas, Markdown and LaTeX.
- Explicit conventions. Averaging, label order, the positive class and zero-division behaviour are stated and recorded in every result, never silently assumed.
- Validated. Each metric is tested against scikit-learn where definitions coincide, plus property-based tests and edge cases.
- Documented. Every metric carries its definition, formula, range, input requirements and references,
available programmatically through
metric_info(). - Efficient.
evaluate()validates inputs once and computes the confusion matrix once for all metrics. - Lightweight. Requires only NumPy, SciPy and pandas.
Quick start
import evalsuite as es
y_true = [0, 1, 1, 0, 1, 0]
y_pred = [0, 1, 0, 0, 1, 1]
y_prob = [0.1, 0.9, 0.4, 0.2, 0.8, 0.6]
result = es.evaluate(y_true, y_pred, y_prob=y_prob)
print(result.summary())
result["f1"] # MetricResult(f1=0.666667)
f"{result['mcc']:.3f}" # '0.333'
result.to_latex(caption="Test-set performance")
result.to_dataframe()
es.f1(y_true, y_pred) # individual metrics
es.roc_auc(y_true, y_prob)
es.metric_info("classification.mcc").formula # documentation
es.list_metrics("regression")
Comparing models
result = es.compare(
y_true,
{"logistic": pred_lr, "forest": pred_rf, "boosting": pred_gb},
probabilities={"logistic": prob_lr, "forest": prob_rf, "boosting": prob_gb},
random_state=0,
)
print(result.summary()) # estimates with 95% CIs, paired tests, Holm-adjusted p-values
result.to_latex(label="tab:models")
es.bootstrap_ci("f1", y_true, y_pred, average="macro", random_state=0) # BCa interval for any metric
es.accuracy_ci(y_true, y_pred) # Wilson interval
es.delong_test(y_true, prob_a, prob_b) # two correlated AUCs
es.mcnemar_test(y_true, pred_a, pred_b)
Every model is evaluated on the same bootstrap resamples, so differences are paired. Accuracy is compared with McNemar's test, binary ROC AUC with DeLong's test and other metrics with a paired bootstrap test; p-values are adjusted for multiple comparisons (Holm by default).
Clinical evaluation
report = es.diagnostic_report(y_true, y_pred) # binary test vs reference standard
print(report)
# Sensitivity, specificity, PPV, NPV (Wilson CIs), LR+ and LR− (log CIs, Simel 1991),
# diagnostic odds ratio (Woolf), Youden's J, accuracy and prevalence
es.lr_positive(y_true, y_pred)
es.youden_j(y_true, y_pred)
dca = es.decision_curve(y_true, {"model": y_prob}) # net benefit vs treat all / treat none
dca.useful_range() # thresholds where the model beats both
es.plot.decision_curve(y_true, {"model": y_prob})
Ratios that divide by zero are inf or NaN with a warning, never 0.
Calibration
es.calibration_report(y_true, y_prob) # Brier, ECE, MCE, intercept, slope, Hosmer–Lemeshow
es.calibration_slope(y_true, y_prob) # ideal 1; < 1 means predictions are too extreme
es.calibration_intercept(y_true, y_prob) # ideal 0 (calibration-in-the-large)
es.hosmer_lemeshow(y_true, y_prob, n_groups=10)
Statistical tests
es.t_test(scores_a, scores_b) # Welch by default; mean difference with CI and Cohen's d
es.paired_t_test(fold_scores_a, fold_scores_b)
es.wilcoxon_test(fold_scores_a, fold_scores_b) # with matched-pairs rank-biserial r
es.mann_whitney_test(a, b) # with rank-biserial r
es.friedman_test(scores_a, scores_b, scores_c) # with Kendall's W
es.kruskal_wallis_test(g1, g2, g3)
es.shapiro_wilk_test(residuals)
es.chi_square_test(table) # with Cramér's V
es.fisher_exact_test([[8, 2], [1, 5]])
es.adjust_pvalues(p_values, method="hochberg") # also holm, bonferroni, bh, by
Every test returns a TestResult with the statistic, p-value and an effect size, computed with SciPy and
checked against SciPy and statsmodels in the test suite.
Segmentation
# label masks: one 2-D image, or images on the first axis (N, H, W) / (N, D, H, W), or a list of masks
es.dice(y_true, y_pred) # macro over classes, pixel counts summed over the dataset
es.iou(y_true, y_pred, average=None) # per class; classes absent from both masks are NaN, not 0
es.miou(y_true, y_pred, ignore_index=255)
es.dice(y_true, y_pred, aggregate="image") # mean of per-image scores (medical imaging convention)
es.boundary_iou(y_true, y_pred) # Cheng et al. 2021
es.hausdorff_distance(y_true, y_pred, percentile=95, spacing=(0.8, 0.8)) # HD95 in mm
report = es.segmentation_report(y_true, y_pred, class_names={0: "background", 1: "liver"})
print(report) # mIoU, Dice, pixel accuracy, Boundary IoU, HD95, ASSD + per-class table
es.plot.segmentation(image, y_true[0], y_pred[0]) # prediction fill, truth outline
es.plot.per_class(report, metric="iou") # per-class bars (also takes a detection report)
Object detection
y_true = [{"boxes": [[x1, y1, x2, y2], ...], "labels": [3, ...]}, ...] # one dict per image
y_pred = [{"boxes": [...], "labels": [...], "scores": [...]}, ...]
report = es.detection_report(y_true, y_pred) # the 12 COCO numbers + AP per class
report["map"], report["map_50"], report["mar_100"]
es.mean_average_precision(y_true, y_pred, iou_threshold=0.5) # mAP@.50
es.average_precision_detection(y_true, y_pred, interpolation="voc") # per class, VOC-style
es.box_iou(boxes_a, boxes_b, box_format="xywh")
y_true, y_pred = es.from_coco("instances_val.json", "detections.json")
es.plot.detection_pr(y_true, y_pred)
The COCO protocol (crowd regions, area ranges, max detections, 101-point interpolation) matches
pycocotools to the last digit in the test suite.
Models can be compared over the same images with intervals and paired tests, as for every other task:
es.compare(y_true, {"unet": masks_a, "deeplab": masks_b}) resamples images; for detection it compares
mAP.
LLM evaluation
es.bleu(references, predictions) # sacreBLEU-identical; also chrf, ter, rouge_l, meteor, cider
es.text_report(references, predictions) # BLEU, chrF(++), TER, ROUGE, METEOR, EM, token F1 at once
es.bertscore(ref_token_embs, pred_token_embs) # from your encoder's embeddings; also mauve, moverscore
es.model_score(refs, preds, scorer=comet_fn) # any learned metric or judge, with intervals and compare
es.faithfulness(claim_verdicts) # also hallucination_rate, citation_recall, answer_correctness
es.bradley_terry(comparisons, scale="elo") # also win_rate, elo_ratings, krippendorff_alpha
es.ndcg_at_k(relevant, retrieved, k=10) # also mrr, context_precision, context_recall
es.json_schema_compliance(outputs, schema) # also tool_call_f1, instruction_compliance_rate
es.pass_at_k(n_samples, n_correct, k=10) # also benchmark_accuracy(..., style="gsm8k")
es.compare(references, {"a": preds_a, "b": preds_b}, metrics=["bleu", "rouge_l"]) # paired over examples
es.plot.ratings(comparisons) # Bradley–Terry leaderboard with intervals; also win_matrix, text_scores
es.spice(reference_tuples, candidate_tuples) # scene-graph F-score; parser= for captions, synonyms="wordnet"
LLM systems
Safety, robustness, uncertainty, agents, multilingual, code, long context and serving cost. The verdicts these take (harmful or not, test passed or not, confidence) come from your own judge, classifier or sandbox; EvalSuite turns them into rates with Wilson intervals and breakdowns. PII detection, CodeBLEU, cyclomatic complexity and the maintainability index (identical to radon) are computed from the text itself, and EvalSuite never executes generated code.
import evalsuite as es
es.harmful_response_rate([True, False, False, True], harmful_prompt=[True, True, False, True])
es.over_refusal_rate(refused=[True, False, True], should_refuse=[True, False, False])
es.pii_leakage_rate(["mail me at a@b.io", "nothing here"]) # e-mail, phone, Luhn-checked cards, IPs
es.paraphrase_consistency([["Paris", "paris"], ["4", "5", "4"]])
es.aurc(correct=[True, True, False, True], confidence=[0.9, 0.8, 0.7, 0.4]) # also selective_risk
es.plan_adherence(plans=[["search", "read", "answer"]], executed=[["search", "answer"]])
es.language_parity({"en": [1, 1, 0, 1], "sw": [1, 0, 0, 1]})
es.codebleu(["def add(a, b):\n return a + b"], ["def add(x, y):\n return x + y"])
es.code_complexity(["def f(x):\n return 1 if x else 0"]) # cyclomatic complexity + maintainability
es.needle_in_haystack(correct=[1, 0, 1, 1], context_lengths=[4000, 4000, 8000, 8000], depths=[0, 0.5, 0, 0.5])
es.time_to_first_token(request_times=[0.0, 1.0], first_token_times=[0.21, 1.35])
es.inference_cost([1200, 800], [300, 150], input_price=3.0, output_price=15.0) # prices per 1M tokens
Classification report
report = es.classification_report(y_true, y_pred)
print(report) # per-class precision, recall, F1, specificity, support + averages
report.save("report.html") # also .csv .md .tex .json .txt
Plots
pip install "evalsuite-python[plot]" # adds matplotlib; importing evalsuite never loads it
es.plot.roc(y_true, {"logistic": prob_lr, "forest": prob_rf}) # AUC in the legend
es.plot.pr(y_true, prob) # AP and the prevalence line
es.plot.calibration(y_true, prob) # reliability diagram, ECE, Brier
es.plot.confusion_matrix(y_true, y_pred, normalize="true")
es.plot.residuals(y_reg, pred_reg) # or kind="predicted"
es.plot.comparison(es.compare(...)) # forest plot with CIs
Each function returns a matplotlib Axes (pass ax= to draw into your own figure). The numbers shown are
computed with EvalSuite's metrics, so plots and tables always agree. Several models get distinct colours
and line styles, so figures stay readable in greyscale print.
Exports
Every result (evaluate, classification_report, compare, single metrics) exports to summary(),
to_json(), to_csv(), to_dataframe(), to_markdown(), to_latex() and to_html(), and save(path) picks
the format from the extension. HTML pages are standalone (inline CSS, no scripts) and escape all text.
Command line
evalsuite evaluate predictions.csv --y-true label --y-pred pred --y-prob prob
evalsuite report predictions.csv --y-true label --y-pred pred -o report.html
evalsuite compare predictions.csv --y-true label --pred lr=pred_lr --pred rf=pred_rf \
--prob lr=p_lr --prob rf=p_rf --plot comparison.png
evalsuite plot roc predictions.csv --y-true label --y-prob prob -o roc.png
evalsuite diagnostic predictions.csv --y-true label --y-pred pred # sensitivity, LR+, DOR... with CIs
evalsuite calibration predictions.csv --y-true label --y-prob prob # slope, intercept, ECE, HL
evalsuite plot decision predictions.csv --y-true label --y-prob prob -o dca.png
evalsuite segmentation true_masks.npy pred_masks.npy --ignore-index 255 --plot per_class.png
evalsuite detection instances_val.json detections.json --plot pr_curves.png
evalsuite metrics --category clinical
evalsuite info classification.mcc
evalsuite benchmark --quick
Input files can be CSV, TSV, Parquet or JSON. Output format follows --format or the -o extension
(text, json, csv, markdown, latex, html). Errors are reported in one line with exit code 2.
Performance
Benchmarked against reference implementations on the same data (fastest of 5 runs; EvalSuite 0.5.1, Python 3.13, NumPy 2.5, Linux x86_64). Overall: 81 of 81 rows agree with the reference (largest difference 2.5e-14), 56 are faster, and the geometric-mean speed-up across all 27 cases and three sizes is 1.89×. At 1,000,000 samples (pixels for segmentation; 1,000 images for detection and 50 for Hausdorff; 10,000 examples for the LLM cases), in alphabetical order:
| Case | Reference | EvalSuite (ms) | Reference (ms) | Speed-up |
|---|---|---|---|---|
| 10 classes: macro F1 | scikit-learn | 41.5 | 137.6 | 3.3× |
| agreement: Krippendorff's alpha, interval (4 raters × 10000 items) | krippendorff | 4.5 | 4.5 | 1.0× |
| binary: 8 label metrics via evaluate() | scikit-learn | 49.4 | 1056.8 | 21.4× |
| binary: ROC AUC | scikit-learn | 200.3 | 338.5 | 1.7× |
| calibration: slope and intercept | statsmodels | 344.3 | 1541.7 | 4.5× |
| clinical: diagnostic report (7 CIs) | statsmodels | 33.5 | 9.6 | 0.29× |
| clinical: sensitivity, specificity, LR+, LR− | scikit-learn | 137.9 | 438.1 | 3.2× |
| decision curve: 99 thresholds | NumPy loop | 172.1 | 236.2 | 1.4× |
| detection: COCO evaluation (1000 images) | pycocotools | 1147.9 | 1351.0 | 1.2× |
| multiple testing: Hochberg (n p-values) | statsmodels | 91.9 | 98.4 | 1.1× |
| regression: MAE, MSE, RMSE, R² via evaluate() | scikit-learn | 32.0 | 22.6 | 0.71× |
| retrieval: MRR, MAP@20, NDCG@10 (10000 queries) | ranx | 248.6 | 897.0 | 3.6× |
| segmentation: Dice and IoU per class (n = pixels) | scikit-learn | 51.6 | 289.5 | 5.6× |
| segmentation: Hausdorff distance (50 images) | SciPy | 31.9 | 23.8 | 0.75× |
| statistics: Cramér's V (5×5 table) | SciPy | 0.3 | 0.3 | 0.95× |
| statistics: Mann–Whitney U | SciPy | 390.3 | 425.8 | 1.1× |
| statistics: Welch t-test | SciPy | 37.7 | 16.3 | 0.43× |
| structured: JSON Schema compliance (10000 documents) | jsonschema | 159.4 | 371.7 | 2.3× |
| systems: CodeBLEU n-gram terms (1000 programs) | codebleu | 740.3 | 791.9 | 1.1× |
| systems: confidence AUROC (n answers) | scikit-learn | 194.4 | 327.7 | 1.7× |
| systems: distribution-shift drop, Welch CI (n scores) | SciPy | 13.3 | 16.9 | 1.3× |
| systems: invalid tool calls vs JSON Schema (10000 tasks) | jsonschema | 145.0 | 333.2 | 2.3× |
| systems: latency p50 / p95 / p99 (n requests) | NumPy | 41.0 | 19.7 | 0.48× |
| systems: maintainability index (1000 programs) | radon | 138.3 | 168.7 | 1.2× |
| text: corpus BLEU and chrF (10000 sentences) | sacreBLEU | 2726.2 | 3121.0 | 1.1× |
| text: METEOR, exact and stem matches (10000 sentences) | NLTK | 507.4 | 590.3 | 1.2× |
| text: ROUGE-1, ROUGE-2, ROUGE-L (10000 sentences) | rouge-score | 1147.3 | 1010.7 | 0.88× |
Every metric is benchmarked too (evalsuite benchmark --suite metrics): all 225 registered metrics and
statistics functions at 10,000 and 100,000 samples. 86 are compared with a reference library, 118 with an
independent textbook formula and 21 (learned, judge-dependent or randomised) are timed alone; all 204
comparisons agree, and against the libraries the geometric-mean speed-up is 2.67× (139 of 172
measurements faster). Per-metric table in BENCHMARKS.md.
evaluate() validates inputs once and builds the confusion matrix once for all metrics, which is where most
of the speed-up comes from. Hypothesis tests use SciPy underneath, so they match its speed at best; rows
below 1× pay for input validation and the extra intervals and effect sizes EvalSuite reports. Reproduce on
your machine with evalsuite benchmark; full table (1k, 100k and 1M samples, peak memory) and notes in
BENCHMARKS.md.
Metrics in this release
Classification (binary, multiclass, multilabel; micro/macro/weighted/samples/per-class averaging; sample weights): accuracy, balanced accuracy, precision, recall, specificity, NPV, F1, F-beta, Jaccard, MCC, Cohen's kappa (unweighted, linear, quadratic), Hamming loss, confusion matrix, ROC AUC (binary, one-vs-rest, one-vs-one), average precision, ROC and PR curves, log loss, Brier score, top-k accuracy, calibration curve and expected calibration error.
Clinical (binary; pos_label; sample weights): sensitivity, specificity, PPV, NPV, positive and
negative likelihood ratios, diagnostic odds ratio, Youden's J, net benefit and decision curves, and a
diagnostic report with confidence intervals for all of them.
Calibration: calibration curve, Brier score, expected and maximum calibration error, calibration slope and intercept, Hosmer–Lemeshow test.
Statistics: confidence intervals (bootstrap percentile/basic/BCa, Wilson, Clopper–Pearson, DeLong), paired tests (McNemar, DeLong, paired bootstrap), t-tests (Welch, Student, paired), Mann–Whitney, Wilcoxon, Kruskal–Wallis, Friedman, Shapiro–Wilk, χ², Fisher's exact; effect sizes (Cohen's d, Hedges' g, Cliff's delta, Cramér's V); multiple-testing corrections (Bonferroni, Holm, Hochberg, Benjamini–Hochberg, Benjamini–Yekutieli).
Segmentation (2-D and 3-D label masks; ignore_index; dataset or per-image aggregation): Dice, IoU,
mIoU, pixel accuracy, mean pixel accuracy, Boundary IoU, Hausdorff distance and HD95, average symmetric
surface distance (with pixel spacing), confusion matrix and a full report.
Object detection: box IoU (xyxy, xywh, cxcywh), COCO mAP@[.50:.95], mAP@.50, mAP@.75, mAP and mAR by object size, AP per class with COCO or VOC interpolation, precision-recall curves, COCO file import.
Regression (single and multi-output; sample weights): MAE, MSE, RMSE, R², adjusted R², MAPE, sMAPE, MSLE, RMSLE, median absolute error, explained variance, max error, mean bias error, quantile (pinball) loss, Huber loss, relative absolute error, relative squared error.
LLM evaluation (0.4.0): BLEU, sentence BLEU, chrF/chrF++, TER, ROUGE-1/2/L/Lsum, METEOR, CIDEr-D, SPICE,
perplexity, cross-entropy, Distinct-n, Self-BLEU, MAUVE, BERTScore, embedding similarity, MoverScore, learned
metrics via model_score; faithfulness, hallucination rate, groundedness, citation precision/recall, claim
verification, knowledge consistency, answer correctness/relevance, abstention, exact match, token F1; win
rate, Bradley–Terry, Elo, Krippendorff's alpha, Fleiss' kappa, judge agreement and biases, rubric scores;
pass@k, majority vote, benchmark accuracy; Precision/Recall/Hit rate@k, MRR, MAP, NDCG, context precision,
recall and relevance; JSON / JSON Schema / XML validity, required fields, tool calls, instruction following.
LLM systems (0.5.0): harmful-response, refusal, over-refusal, attack-success and red-team success rates, expected maximum toxicity, CrowS-Pairs stereotype preference, WEAT, PII leakage, memorization exposure, policy violations; adversarial, typo-noise, OOD and code-switching robustness, distribution-shift drop, paraphrase / counterfactual / cross-lingual consistency, response stability, contradiction rate, failure and recovery rates, prompt sensitivity, truncation sensitivity; adaptive calibration error, risk-coverage curve, AURC and E-AURC, selective risk, risk at coverage, coverage at risk, confidence AUROC; task completion, invalid tool calls, tool-use efficiency, steps per task, plan adherence, state tracking, tool-failure recovery, loop rate, human intervention, cost per task; language ID, bitext mining, language parity, direct assessment, cultural appropriateness, language consistency; unit-test pass rate, syntax validity, static analysis and security findings, execution success, test coverage, patch acceptance, SWE-bench resolved rate, CodeBLEU, cyclomatic complexity and maintainability index, runtime efficiency; long-context accuracy by length, needle-in-a-haystack grid, position accuracy, lost-in-the-middle gap, context utilization, summary coverage, compression ratio, citation coverage, cross-document consistency; TTFT, TPOT, latency percentiles, throughput, token usage, cost, resource utilization, energy per request, requests per second, availability.
Conventions
average="auto"resolves to"binary"for binary targets and"macro"otherwise; the resolved value is stored inresult.params["average"].- Labels are sorted unless you pass
labels=[...]; that order defines per-class outputs and the columns of 2-Dy_prob. - Undefined ratios (zero denominators) return 0 with an
UndefinedMetricWarning; passzero_division=np.nanto propagate NaN, or0/1to choose silently. - Domain violations raise clear errors instead of being patched over (for example MAPE with zero targets).
Development
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
pytest --cov=evalsuite
ruff check . && ruff format --check . && mypy
Links
- PyPI: https://pypi.org/project/evalsuite-python/
- Website and documentation: https://evalsuite-nine.vercel.app
- Website source: https://github.com/mkcs28/evalsuite
Credits
Authors and maintainers: Manoj Kumar C S and Nikhil D Bharadwaj.
License
MIT. See LICENSE.
Metadata
Release files for evalsuite-python 0.5.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| evalsuite_python-0.5.2.tar.gz | 305.0 kB | With provenance | Details for evalsuite_python-0.5.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| evalsuite_python-0.5.2-py3-none-any.whl | Python 3 | none | any | Details for evalsuite_python-0.5.2-py3-none-any.whl |
Total release size: 578.4 kB
Release files / evalsuite_python-0.5.2.tar.gz
| Download URL | evalsuite_python-0.5.2.tar.gz |
|---|---|
| Size | 305.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
368d268e15e9baf4d6ec3276b8b705b5259a345bd159cf2accb90db0b2bc58ce
|
|
BLAKE2b-256 checksum How to use checksums |
b2788769359a8b179a57b2ad1c1105bee3e9dd3c887e050d08c72d19f2cb8d40
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 9, 2026.
Transparency logRelease files / evalsuite_python-0.5.2-py3-none-any.whl
| Download URL | evalsuite_python-0.5.2-py3-none-any.whl |
|---|---|
| Size | 273.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
6164a93f4a868a3a501850a9ce3abf9c2083b9bbbbf037207ddf7469bfb718fa
|
|
BLAKE2b-256 checksum How to use checksums |
1d48b168f4369b69e3bcbc7df9c5c91a73752b476b28e208b1281da6d1fc332f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 9, 2026.
Transparency logSecurity for evalsuite-python 0.5.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for evalsuite_python-0.5.2.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 9, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.