This release is a pre-release and may not be stable for production use.
Verifiers: Environments for LLM Reinforcement Learning
Documentation • Environments Hub • PRIME-RL
News & Updates
- [05/07/26] v0.1.14 is released, featuring the v1 Taskset/Harness API, shared eval and training config shape, model-family starter configs, OpenAI Responses and renderer-backed clients, per-turn timing, GEPA prompt artifacts, Lean guard markers, and release/infrastructure hardening.
- [04/28/26] v0.1.13.dev8 is released, featuring per-rollout wall-clock timeouts for
MultiTurnEnv, CLI timeout config, sandbox timeout propagation, and smallerCliAgentEnvand RLM fixes. - [04/17/26] v0.1.12 is released, featuring upstreamed opencode and RLM harnesses/tasksets, major
RLMEnvimprovements (context dropping, prompt builder, hardened transport), multi-worker env server support, expandedvf-tuicapabilities, and richer eval configuration. - [03/12/26] v0.1.11 is released, featuring a unified client stack, major
RLMEnvand env server reliability improvements, a substantially refined eval TUI, new pass@k and ablation sweep support, and bundled opencode environments. - [02/10/26] v0.1.10 is released, featuring OpenEnv and BrowserEnv integrations, resumed evals, improved rollout and token tracking, safer sandbox lifecycle behavior, refreshed workspace setup, and opencode harbor improvements.
- [01/08/26] v0.1.9 is released, featuring a number of new experimental environment class types, monitor rubrics for automatic metric collection, improved workspace setup flow, improved error handling, bug fixes, and a documentation overhaul.
- [11/19/25] v0.1.8 is released, featuring a major refactor of the rollout system to use trajectory-based tracking for token-in token-out training across turns, as well as support for truncated or branching rollouts.
- [11/07/25] Verifiers v0.1.7 is released! This includes an improved quickstart configuration for training with prime-rl, a new included "nano" trainer (
vf.RLTrainer, replacingvf.GRPOTrainer), and a number of bug fixes and improvements to the documentation. - [10/27/25] A new iteration of the Prime Intellect Environments Program is live!
Overview
Verifiers is our library for creating environments to train and evaluate LLMs.
Environments contain everything required to run and evaluate a model on a particular task:
- A dataset of task inputs
- A harness for the model (tools, sandboxes, context management, etc.)
- A reward function or rubric to score the model's performance
Environments can be used for training models with reinforcement learning (RL), evaluating capabilities, generating synthetic data, experimenting with agent harnesses, and more.
Verifiers is tightly integrated with the Environments Hub, as well as our training framework prime-rl and our Hosted Training platform.
Getting Started
Ensure you have uv installed, as well as the prime CLI tool:
# install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
# install the prime CLI
uv tool install prime
# log in to the Prime Intellect platform
prime login
To set up a new workspace for developing environments, do:
# ~/dev/my-lab
prime lab setup
This sets up a Python project if needed (with uv init), installs verifiers (with uv add verifiers), creates the recommended workspace structure, and downloads useful starter files:
configs/
├── endpoints.toml # OpenAI-compatible API endpoint configuration
├── rl/ # Example configs for Hosted Training
├── eval/ # Example multi-environment eval configs
└── gepa/ # Example configs for prompt optimization
.prime/
└── skills/ # Bundled workflow skills for create/browse/review/eval/GEPA/train/brainstorm
environments/
└── AGENTS.md # Documentation for AI coding agents
AGENTS.md # Top-level documentation for AI coding agents
CLAUDE.md # Claude-specific pointer to AGENTS.md
Alternatively, add verifiers to an existing project:
uv add verifiers && prime lab setup --skip-install
Environments built with Verifiers are self-contained Python modules. To initialize a fresh environment template, do:
prime env init my-env # creates a new template in ./environments/my_env
Add an explicit harness loader when the environment owns harness behavior:
prime env init my-env --with-harness
For OpenEnv integration, use:
prime env init my-openenv --openenv
Then copy your OpenEnv project into environments/my_openenv/proj/ and build the image with:
uv run vf-build my-openenv
This will create a new module called my_env with a basic environment template.
environments/my_env/
├── my_env.py # Main implementation
├── pyproject.toml # Dependencies and metadata
└── README.md # Documentation
Environment modules should expose a load_environment function which returns an
environment object. For simple legacy environments, this can still be a direct
constructor:
# my_env.py
import verifiers as vf
def load_environment(dataset_name: str = 'gsm8k') -> vf.Environment:
dataset = vf.load_example_dataset(dataset_name) # 'question'
async def correct_answer(completion, answer) -> float:
completion_ans = completion[-1]['content']
return 1.0 if completion_ans == answer else 0.0
rubric = vf.Rubric(funcs=[correct_answer])
env = vf.SingleTurnEnv(dataset=dataset, rubric=rubric)
return env
The same environment package is the unit used by evals and prime-rl. The
trainer owns model, endpoint, sampling, and rollout count; v1-specific options
stay on the taskset or harness config that owns them:
# configs/rl/my-v1-env.toml
model = "Qwen/Qwen3-30B-A3B-Instruct-2507"
max_steps = 100
batch_size = 256
rollouts_per_example = 8
[sampling]
max_tokens = 4096
[[env]]
id = "my-env"
[env.harness]
max_turns = 1
[env.taskset]
system_prompt = "Reverse text exactly."
[env.taskset.scoring.contains_answer]
weight = 1.0
prime env install my-env
For self-managed training launch commands, use the prime-rl documentation.
To run a local evaluation with any OpenAI-compatible model, do:
prime eval run my-env -m openai/gpt-5-nano # run and save eval results locally
Evaluations use Prime Inference by default; configure your own API endpoints in ./configs/endpoints.toml.
View local evaluation results in the terminal UI:
prime eval view
To publish the environment to the Environments Hub, do:
prime env push --path ./environments/my_env
To run an evaluation directly from the Environments Hub, do:
prime eval run primeintellect/math-python
Documentation
Environments — Create datasets, rubrics, and custom multi-turn interaction protocols.
Evaluation - Evaluate models using your environments.
Training — Train models in your environments with reinforcement learning.
Development — Contributing to verifiers
API Reference — Understanding the API and data structures
FAQs - Other frequently asked questions.
Citation
Originally created by Will Brown (@willccbb).
If you use this code in your research, please cite:
@misc{brown_verifiers_2025,
author = {William Brown},
title = {{Verifiers}: Environments for LLM Reinforcement Learning},
howpublished = {\url{https://github.com/PrimeIntellect-ai/verifiers}},
note = {Commit abcdefg • accessed DD Mon YYYY},
year = {2025}
}
1 maintainer
Metadata
1 maintainer
Release files for verifiers 0.1.15.dev359
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| verifiers-0.1.15.dev359.tar.gz | 778.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| verifiers-0.1.15.dev359-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.6 MB
Release files / verifiers-0.1.15.dev359.tar.gz
| Download URL | verifiers-0.1.15.dev359.tar.gz |
|---|---|
| Size | 778.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
db1b8249f029b4572b1cf48cc3ac3ea60b285cc25ee293bb15679268b2a145cb
|
|
BLAKE2b-256 checksum How to use checksums |
b4923420ebf2f08cbaafc6d76660c0a8b6bd2a9e14d951c46bc2d7f3ceb03c30
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jun 23, 2026.
Transparency logRelease files / verifiers-0.1.15.dev359-py3-none-any.whl
| Download URL | verifiers-0.1.15.dev359-py3-none-any.whl |
|---|---|
| Size | 802.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
cf4ccfaaa393e46971fd2956b41c5e08cd515d15a91ccbdd4b4839489e344ab3
|
|
BLAKE2b-256 checksum How to use checksums |
fcd1a12754090a42f53b9c5806f7df021045ff311e592947244eeab5aab3d2ef
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jun 23, 2026.
Transparency log
