sovos-provebench

The bench suite that measures SOVOS substrate against external benchmarks on a per-axis basis. Brought together from kaggle/gspc_axes/ (Kaggle-targeted task cards) plus the top-level provbench_*.py scripts into one canonical package.

Provenance

Absorbed 2026-08-11:

Files

All files preserve their byte-content — copied line-for-line from the original locations.

Run

The bench tasks are intended to be invoked from a Kaggle kernel or from the substrate's verification script. Use either pattern:

import sys; sys.path.insert(0, "SOVOS/packages/sovos-provebench")
import govbench_task
import provbench_task
# ...

Or run the scripts standalone:

python3 SOVOS/packages/sovos-provebench/provbench_15asset_rerun.py

Dependencies

These scripts were originally Kaggle-notebook-bound. All 6 task cards import kaggle_benchmarks — a Kaggle-platform-provided module that ships only inside Kaggle notebooks. As a consequence:

The two top-level provbench_*.py scripts (provbench_15asset_rerun.py, provbench_canonical_bound.py) do not import kaggle_benchmarks — they read prior bench output from disk and recompute. They work off-Kaggle as long as the output JSON from a prior Kaggle run is on disk.

This package is therefore two-faced: the provbench_*.py scripts are run-anywhere tools; the 6 task cards are Kaggle-bound instruments preserved verbatim for historical + reproducibility reasons. None of the provebench bench tasks are required for any other SOVOS package — the live measurement surface is sovos-arena, which uses its own probe bank (GSPC_AXES, PROBE_BANK).

Cross-link

These task cards are the input to sovos-arena. The arena runs each probe-bank through a target system and produces the Wilson-CI per-axis profile; this package supplies the probes + scorers for the canonical bench suite.