A collection of MedARC utilities and tools for Prime Intellect's verifiers package
Project description
medarc-verifiers
Utilities and CLI for running medical LLM benchmarks with verifiers. Provides batch orchestration, result processing, and shared building blocks for authoring environments.
Install
pip install medarc-verifiers
Environments are installed separately via prime env install <owner/env> (from the Prime Intellect Hub) or vf-install <env> (from a local directory).
medarc-eval
medarc-eval covers the full evaluation pipeline:
| Command | Description |
|---|---|
medarc-eval <ENV> |
Run a single benchmark; env-specific flags inferred from load_environment() |
medarc-eval bench |
Run multiple model × environment jobs from a YAML config, with resume support |
medarc-eval process |
Convert raw outputs to analysis-ready parquet |
medarc-eval winrate |
Compute HELM-style win rates across models |
See medarc-eval.md for full documentation.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file medarc_verifiers-0.1.0.tar.gz.
File metadata
- Download URL: medarc_verifiers-0.1.0.tar.gz
- Upload date:
- Size: 141.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: uv/0.10.4 {"installer":{"name":"uv","version":"0.10.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
0694dcb45ee8ab2e5fc5c182a16a278f31503d2d2774c3e183e6f82be832901b
|
|
| MD5 |
1a5bab40faa30f1b56601233fff6476e
|
|
| BLAKE2b-256 |
f768ce99b5f806169fe04522975a4c7524d4296eb11baa1e71dc0e7d5ee9c325
|
File details
Details for the file medarc_verifiers-0.1.0-py3-none-any.whl.
File metadata
- Download URL: medarc_verifiers-0.1.0-py3-none-any.whl
- Upload date:
- Size: 176.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: uv/0.10.4 {"installer":{"name":"uv","version":"0.10.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
34c4e7bb2a539d5ff3c62d1b218e14a9e3cb652613fb179b4e0547f06292a972
|
|
| MD5 |
2a7946611d88e7bfe5d8429dcca61b54
|
|
| BLAKE2b-256 |
900c0521b20c980d7a936383d15d9d8ca51d61efae93c119277c8d56ceebeb9c
|
