medarc-verifiers
Utilities and CLI for running medical LLM benchmarks with verifiers. Provides batch orchestration, result processing, and shared building blocks for authoring environments.
Install
pip install medarc-verifiers
Environments are installed separately via prime env install <owner/env> (from the Prime Intellect Hub) or vf-install <env> (from a local directory).
medarc-eval
medarc-eval covers the full evaluation pipeline:
| Command | Description |
|---|---|
medarc-eval <ENV> |
Run a single benchmark; env-specific flags inferred from load_environment() |
medarc-eval bench |
Run multiple model × environment jobs from a YAML config, with resume support |
medarc-eval process |
Convert raw outputs to analysis-ready parquet |
medarc-eval winrate |
Compute HELM-style win rates across models |
See medarc-eval.md for full documentation.
1 maintainer
Metadata
1 maintainer
Release files for medarc-verifiers 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| medarc_verifiers-0.1.0.tar.gz | 141.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| medarc_verifiers-0.1.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 317.7 kB
Release files / medarc_verifiers-0.1.0.tar.gz
| Download URL | medarc_verifiers-0.1.0.tar.gz |
|---|---|
| Size | 141.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0694dcb45ee8ab2e5fc5c182a16a278f31503d2d2774c3e183e6f82be832901b
|
|
BLAKE2b-256 checksum How to use checksums |
f768ce99b5f806169fe04522975a4c7524d4296eb11baa1e71dc0e7d5ee9c325
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.10.4 {"installer":{"name":"uv","version":"0.10.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / medarc_verifiers-0.1.0-py3-none-any.whl
| Download URL | medarc_verifiers-0.1.0-py3-none-any.whl |
|---|---|
| Size | 176.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
34c4e7bb2a539d5ff3c62d1b218e14a9e3cb652613fb179b4e0547f06292a972
|
|
BLAKE2b-256 checksum How to use checksums |
900c0521b20c980d7a936383d15d9d8ca51d61efae93c119277c8d56ceebeb9c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.10.4 {"installer":{"name":"uv","version":"0.10.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
