5 projects
rolloutwatch
Did my prompt or model change make things worse? Watch LLM quality after a change, checking as often as you like without false alarms
evalnoise
Where does the noise in an eval score come from? Split it across questions, judges, prompt wordings and runs, and plan where to spend the next API calls
costplan
How much will my agent cost? Plan AI agent budgets, caps and spend from a handful of pilot runs
sturdytools
Retries, timeouts, rate limits and circuit breakers for AI agent tools
evalsize
How many eval examples do I need? Sample size and power planning for LLM evals