whileai 0.89

MID-TRAINING AND POST-TRAINING FOR LANGUAGE MODELS

Building RL and SFT datasets for agents is hard. whileai is the library that does it, and that measures whether training on them worked. Give it an agent, or just the agent's tools and system prompt. It writes the situations the agent might meet, runs the agent through them against a fake world that fails on purpose, and hands back every conversation as a row. You grade the rows with your own judge or a verifier. The package then does the bookkeeping that is easy to skip and expensive to get wrong: pass rates with intervals, difficulty bands for RL, a check that your judge agrees with people, decontamination against your eval set, and a scan for rewards the policy can game. Every method says where it comes from (References).

import whileai as wai


@wai.tool
def get_order(order_id: str) -> dict:
    """Look up an order by id."""  # the function is the tool; its signature is the schema
    ...


wai.configure(agent=wai.OpenAI("gpt-4.1-mini"), judge=wai.Anthropic("claude-haiku-4-5"))

data = wai.simulate(tools=[get_order], system_prompt=POLICY, mode="rl", repeats=8)
scored = data.grade(wai.Judge(rubric=RUBRIC))  # or a verifier, or any callable
print(scored.pass_at)  # pass@1 0.61 [0.54..0.68] | pass@8 0.93 | headroom 0.32
print(wai.judge_trust(scored.rows))  # does the judge agree with people
rows = scored.select(mode="rl")  # the 20..80% band, unanimous groups dropped
rows.export("train.jsonl")  # or rows.push("my-agent-rl-v1")

uv add whileai

Or pip install whileai. Python 3.10 to 3.13, two dependencies, typed. import whileai takes under 200 ms and never touches the network. This package used to be called zeroproof; that name still installs it.

Two domains, kept apart. import whileai as wai is the library: simulate, grade, measure, select, export, on your machine against your models, no account needed. whileai.platform is the While platform: sign in, push datasets, train and serve on hosted GPUs, track versions. Everything that talks to withwhile.com lives there and nowhere else.

Your model, your key

Every role in a run is a model behind an endpoint. Say which with a backend object; its repr tells you where the call goes and which key it uses.

wai.OpenAI("gpt-4.1-mini")  # OpenAI(model='gpt-4.1-mini', key=OPENAI_API_KEY)
wai.OpenAI("gpt-4.1-mini", api_key="sk-...")  # key=given, kept for every OpenAI call
wai.Anthropic("claude-haiku-4-5")  # key=ANTHROPIC_API_KEY
wai.Endpoint("Qwen/Qwen3-4B", url="http://localhost:8000/v1")  # vLLM, SGLang, TGI; key=none needed
wai.Ollama("llama3")  # key=none needed
wai.Hosted()  # the model While hosts, on `whileai login`

wai.configure(agent=, judge=, simulator=, api_key=) sets the process once. A keyword on the call wins over it, with wai.context(...) wins inside the block, and the environment is read only after all three. print(wai.settings) says what each role resolves to. With nothing configured, every role uses the model While hosts on the key from whileai login, and the judge is a different model family from the agent.

The spec strings the objects stand for ("openai:<model>", "anthropic:<model>", "vllm:<model>@<url>", "ollama:<model>") work anywhere a backend does, and OPENAI_BASE_URL points OpenAI at a compatible server. A tool is a typed function under @wai.tool: the signature is the schema, the docstring the description, Annotated[str, "note"] or an Args: block the parameter notes. The mock world answers the calls, faults first; execute=wai.Tool.dispatch([get_order]) has the bodies answer instead. Raw schema dicts still work. No tools at all yet? wai.simulations.draft_tools("a support agent that looks up orders and issues refunds") drafts them on the agent's key. Three things reach While, and only when you ask: leaving the agent on Hosted(), the hosted situation writer, and whileai.platform. whileai status prints which key the SDK will use and where it came from.

Two ways in

You only want evals. Plenty of teams cannot train and still need to know whether the last prompt edit helped. Run whileai init-evals in your project. It finds your agent, writes a judge and a runner around it, and gives you a pass rate with a 95% interval, a table of where the agent fails, and a test that goes red in CI when it gets worse. coverage_gap tells you which situations your tests never reach. compare_runs reruns the same tasks after a prompt or tool change and says whether the change helped. Start at docs.withwhile.com/evals.

You want to train. Grade the same rows, keep the ones that carry signal, export to your trainer. That is the rest of this page. The platform at the end is where hosted training lives, if you want it.

Sixty seconds, offline

No key, no network. seeded_agent is a stand-in agent. It answers honestly most of the time and, on a labeled fraction of rollouts, does one thing wrong on purpose: hedges, flatters, or claims success after a tool failed. Each row records what it did in seeded, so you can check that your judge catches exactly those rows before you trust it on real ones.

import whileai as wai


@wai.tool
def get_order(order_id: str) -> dict:
    """Look up an order by id."""
    ...


data = wai.simulate(
    wai.seeded_agent([get_order]),
    tools=[get_order],
    system_prompt="Help customers with orders.",
    simulator=False,  # situations from templates, no model
    mode="rl",
    repeats=4,
    repeat_policy="fixed",
    budget=64,
)
scored = data.grade(judge=lambda row: {"reward": int(not row["seeded"])})
print(scored.pass_at)
rows = scored.select(mode="rl")
print(rows)

pass@1 0.67 [0.55..0.78] | pass^4 (pass_pow_k) 0.19 [0.00..0.38] | pass@4 1.00 [1.00..1.00] | headroom 0.33 (16 groups, k=4)
rl selection: kept 27 of 64 rows
  band 20%..80% pass rate: 0 asks dropped (0 too easy, 0 too hard)
  unanimous groups dropped: 6; duplicates dropped: 27; truncated drop: 0
  privileged leaks dropped: 4
  groups kept: 10
  hack scan: train

pass@1 is the pass rate over tasks with a bootstrap interval. pass^4 is how often all four rollouts of a task pass. Headroom is pass@4 minus pass@1, the gap an RL update could close. select prints what each gate dropped and why (the warnings that follow are cut here; the quickstart shows them all); rows.export(path) writes them trainer-ready.

To use your own agent, pass any callable that takes the user message and returns {"steps": [...], "final_text": "..."}, or a backend object from the section above. wai.Judge(rubric=...) is the LLM judge as an object; data.grade(spec="typesafe:jev-latest") grades with TypeSafe's Jev instead: typed questions, a probability on every verdict, no output tokens (judge only, on TYPESAFE_API_KEY). Next: docs.withwhile.com/get-started/quickstart.

The loop

The first name in each row is at the top level, wai.<name>. Everything else is one dot down at wai.simulations.<name>, and the science behind each is on docs.withwhile.com/concepts/engine.

The science

SFT. optimize(mode="sft") is rejection sampling [14], [16]: keep the best-scoring completion for each prompt, with a random selector alongside so you can tell whether picking the best did anything. Exported rows carry a loss_mask per message, so the trainer learns from the agent's turns and not from tool output. unroll=True splits a long conversation into one sample per agent turn, each with the context that turn actually saw. format="trl" is the shape SFTTrainer loads [1, ch. 4].

RL with verifiable rewards. When a program can check the answer, the reward should be that program [5]: MathEqual, CodeExec against hidden tests, JSONSchema, and combinations of them. In mode="rl" every prompt gets two rollouts first. Only prompts where those two disagree are filled to k, because a group that all passes or all fails has zero advantage under GRPO [19]. That is DAPO's dynamic sampling [12], applied while the rollouts are generated instead of after. optimize(mode="rl") then keeps the prompts the policy solves 20 to 80% of the time [13] and lets you choose what happens to rollouts that hit the length cap [12]. export_environment writes the tasks, the fake world and the reward as a verifiers package you can hand to a trainer. Rows keep their sampling logprobs so the trainer can form the importance ratio [21], and mean_kl measures drift from the reference model [22].

Character training. Write down how the model should talk as a constitution [23], [24]. load_spec hashes it into spec.version, so an edit to one principle is a new version. The judge is checked against the labels the spec itself carries before it grades anything. Preference pairs are matched on length [7], so the model learns the trait and not "longer is better". Put spec.behaviors() in must_not_regress and delta_report fails any run that improved one trait by giving up another. docs/character-training.md.

Evaluation. Intervals are bootstrapped over tasks, not rollouts, because rollouts of the same task are not independent [8], [10], [11]. Run an eval three times with runs=3 and delta_report refuses to call a change real when it sits inside twice the run-to-run standard deviation (budget is per run: runs=3, budget=100 is up to 300 rows). holdout_size says how many prompts you need to see a given gain at 80% power [11]; most evals are too small. decontaminate checks training rows against the eval set with the 80% n-gram overlap rule [16], and with embeddings when you pass an embedder. docs/evals.md.

Over-optimization. The reward is a proxy for what you want, and RL finds the gap between the two [17]. hack_scan looks for the feature that predicts reward within a task, against a shuffled baseline, so a judge that pays for a phrase or a delimiter shows up before you train on it. judge_probes tries the tricks a policy finds first, flattery included [18]. delta_report(proxy=, target=) fails when the training reward went up and the metric you care about did not. HackMonitor runs the same scan inside a TRL training loop and can stop it. docs/reward-hacking.md.

Recipes

Each recipe is one script and a README that says what you learn, what you need, and how long it takes. All of them run in CI.

The platform

Separate from the library, and optional. Sign in once and the same rows push to an account, train on hosted GPUs, and come back as an OpenAI-compatible endpoint. Everything above this heading runs without it.

From a terminal, for a coding agent that manages the account:

whileai login                    # or: whileai signup --email you@example.com
whileai agents                   # what is tracked, what each one serves
whileai agent refund-bot         # record, behaviors, verdict
whileai runs refund-bot          # the version table
whileai verdict refund-bot       # does the candidate beat the served version, and is it real
whileai promote refund-bot v4    # usually the person's button on the platform
whileai live refund-bot --day 2026-09-17 --version v3 --replies 2400 --flagged 98
whileai keys                     # names and prefixes; create or revoke under Account

Every command takes --json. They are thin calls into whileai.platform. push refuses RL data with no mixed groups, since a trainer would learn nothing from it.

from whileai import platform

platform.login()  # once; or wai.configure(api_key="zp_...")
v1 = rows.push("refunds-v1", holdout=0.2)  # the selection, gated
run = platform.train(v1["datasetId"], method="grpo", steps=200)  # sft | grpo | dpo | rm
run.wait()
model = platform.serve("refunds-v2", run)  # OpenAI-compatible endpoint

If you train with your own code, platform.TrainerCallback reports into the same run page. Traces from production come back through traces=, which points the next simulation at the situations that failed.

Already ran your own eval? Send the rows you scored and the platform groups them into prompts, so you can see what is worth training on.

whileai.send_runs(rows, agent="refunds")  # {scenario_id, prompt, final_text, reward}
print(wai.format_cuts(wai.cuts("refunds"), agent="refunds"))

Documentation

This README is the shape of the loop. The docs are the depth, in the same order:

What we believe and where each belief is enforced is CONSTITUTION.md. The coding standard the package is held to, PyTorch and DSPy ergonomics, is docs/reference/style.md. CHANGELOG.md has one entry per release.

Development

uv sync --extra dev
uv run pytest
uv run ruff check . && uv run mypy && uv run ty check

CI runs the suite on Python 3.10 to 3.13, gates coverage at 90%, and runs every recipe's smoke.sh. CONTRIBUTING.md.

Cite

@software{weiss2026whileai,
  title  = {whileai: post-training data and evaluation for tool-using agents},
  author = {Weiss, Jacob},
  year   = {2026},
  url    = {https://github.com/whilehq/whileai-sdk}
}

References

1. Lambert, N. Reinforcement Learning from Human Feedback. arXiv:2504.12501, 2025.
2. Kuhn, D. R., Wallace, D. R., Gallo, A. M. Software Fault Interactions and Implications for Software Testing. IEEE TSE 30(6), 2004.
3. Yao, S. et al. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv:2406.12045, 2024.
4. Ouyang, L. et al. Training Language Models to Follow Instructions with Human Feedback. NeurIPS, 2022.
5. Lambert, N. et al. Tülu 3: Pushing Frontiers in Open Language Model Post-Training. arXiv:2411.15124, 2024.
6. Cohen, J. A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement 20(1), 1960.
7. Zheng, L. et al. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS, 2023.
8. Chen, M. et al. Evaluating Large Language Models Trained on Code. arXiv:2107.03374, 2021.
9. Wilson, E. B. Probable Inference, the Law of Succession, and Statistical Inference. JASA 22(158), 1927.
10. Efron, B., Tibshirani, R. J. An Introduction to the Bootstrap. Chapman & Hall, 1993.
11. Miller, E. Adding Error Bars to Evals. arXiv:2411.00640, 2024.
12. Yu, Q. et al. DAPO: An Open-Source LLM Reinforcement Learning System at Scale. arXiv:2503.14476, 2025.
13. He, J. et al. Skywork Open Reasoner 1 Technical Report. arXiv:2505.22312, 2025.
14. Yuan, Z. et al. Scaling Relationship on Learning Mathematical Reasoning with Large Language Models. arXiv:2308.01825, 2023.
15. Rafailov, R. et al. Direct Preference Optimization. NeurIPS, 2023.
16. Touvron, H. et al. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288, 2023.
17. Gao, L., Schulman, J., Hilton, J. Scaling Laws for Reward Model Overoptimization. ICML, 2023.
18. Sharma, M. et al. Towards Understanding Sycophancy in Language Models. ICLR, 2024.
19. Shao, Z. et al. DeepSeekMath. arXiv:2402.03300, 2024.
20. DeepSeek-AI. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948, 2025.
21. Schulman, J. et al. Proximal Policy Optimization Algorithms. arXiv:1707.06347, 2017.
22. Ziegler, D. M. et al. Fine-Tuning Language Models from Human Preferences. arXiv:1909.08593, 2019.
23. Bai, Y. et al. Constitutional AI: Harmlessness from AI Feedback. arXiv:2212.08073, 2022.
24. OpenAI. Model Spec, 2024. model-spec.openai.com.

License

Apache-2.0