GOESB Runner

The official, reproducible benchmark runner. It loads a signed Benchmark Profile and Benchmark Pack, captures the full hardware/software/model environment, runs the benchmark (batch / streaming / conversation) against a pluggable runtime adapter, computes metrics, and emits a signed, hash-verifiable result.

The runner never executes arbitrary user code (see the security model).

Quick start

pip install -e ".[dev,faster-whisper]"
goesb --help
goesb env          # print captured environment fingerprint
goesb validate profiles/whisper-medium-en-batch/profile.yaml
goesb validate packs/librispeech-en/pack.yaml

# Fetch this pack's audio (not committed to git — FR-3.5) before running
# against it. Reruns are safe with --skip-download.
python ../scripts/fetch_librispeech_subset.py

# Run a batch benchmark end-to-end: environment capture, transcription,
# normalization + WER/CER/RTF/CPU/RAM, a signed+hashed result on disk.
goesb run whisper-medium-en-batch librispeech-en --repeats 2

Pass --model-override tiny (or base) to goesb run for a fast local smoke test instead of downloading the full whisper-medium the profile specifies — results from an override are for pipeline verification only, never comparable to the official profile's numbers.

Schema validation currently requires running from within this monorepo checkout (see oesb_runner.schema_validation); standalone packaging of the schemas is tracked for M2.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.9.26)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for taktx from gravatar.com taktx

Requires

Python >=3.11

Provides Extra

dev faster-whisper mozilla-data-collective vosk whisper-cpp

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

goesb_runner-0.8.1.tar.gz (99.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

goesb_runner-0.8.1-py3-none-any.whl (85.5 kB view details)

Uploaded Python 3

File details

Details for the file goesb_runner-0.8.1.tar.gz.

File metadata

  • Download URL: goesb_runner-0.8.1.tar.gz
  • Upload date:
  • Size: 99.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for goesb_runner-0.8.1.tar.gz
Algorithm Hash digest
SHA256 e0f82fa156be28d1fe71b2d4a4b698be5e0fc27c5ce4d521d3ad887dabe0fcb0
MD5 97c43f9f36e7edacf4d6bcaa6b865b2d
BLAKE2b-256 db85c7b1a4129b72519608d90286a7a2cc43a4fd7d8c4417a7ef8f902c0e2401

See more details on using hashes here.

Provenance

The following attestation bundles were made for goesb_runner-0.8.1.tar.gz:

Publisher: release.yml on taktx-io/GOESB

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file goesb_runner-0.8.1-py3-none-any.whl.

File metadata

  • Download URL: goesb_runner-0.8.1-py3-none-any.whl
  • Upload date:
  • Size: 85.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for goesb_runner-0.8.1-py3-none-any.whl
Algorithm Hash digest
SHA256 54b71ee4d8452055a43b993dd3c0ef0d296f968f64b3f1d18aaf80c59c65374f
MD5 73639652ea1ca936d7e67a8fbc81e50c
BLAKE2b-256 cfe8141c93af2f027e042b34d1c2a17bd389f07cc8c1efa1f6016f915c95abb4

See more details on using hashes here.

Provenance

The following attestation bundles were made for goesb_runner-0.8.1-py3-none-any.whl:

Publisher: release.yml on taktx-io/GOESB

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page