zeroproof

The ZeroProof Python SDK. One package, two importable modules:

• zeroproof: the platform client. OTLP trace ingest and trace-dataset listing against the token gate.
• zeroproof.simulations: post-training data for an agent. (Was the separate top-level package zeroproof_simulations; that name still imports for two releases with a deprecation warning.) Give it the agent's traces, or its tools and system prompt; it simulates the situations, the people, and the world, plays the agent through multi-turn tool-calling conversations, and returns rows for your grader.

This repo absorbed the zeroproof-simulations package; zeroproof-simulations on PyPI is deprecated in favor of zeroproof.

Releases of zeroproof before 0.3 were an unrelated encrypted agent-to-agent messaging client. That code was removed in 0.04; pin zeroproof<0.3 if you still depend on it.

Two ways in, one engine. Give it the agent's tools and system prompt and it samples situations across everything that agent can be asked. Give it graded traces as well and it aims the budget at the situations that fail in production, so new rows land where the agent is weak and carry both the failure and the fixed version. Every row is a full conversation: user turns, agent turns, tool calls, tool results, scheduled faults. Rows come back ungraded; your grader decides what good means. Default explore: one unique situation per row. How it thinks: docs/simulations.md.

Overview

simulate() is a pipeline.

1. Read the agent. Tools and system prompt. That is the spec of the world.
2. Build a fake world from those tools. Objects, plausible results, and faults (timeout, deny, junk).
3. Write users. A separate writer (same hosted model, different prompt, no agent policy) samples situations across tools, stance, history, and so on.
4. Pick the diverse ones. Embeddings plus a bit of noise so the batch is not 200 copies of the same prompt.
5. Play the agent. It talks, calls tools, gets results, talks again. All of that is stored: user text, agent text, tool calls, tool results, final_text.
6. Grade. Rows come back ungraded. Grade after with zps.grade(...), pass your own grader=, or grade=True for the deterministic conduct score.

Stop when the row cap or the clock hits.

How to use

pip install zeroproof   # or: uv add zeroproof

Bring your own model. Any OpenAI-compatible chat endpoint that returns tool calls works; it writes the situations and plays the agent, so both run on your key:

export OPENAI_API_KEY=...
export OPENAI_BASE_URL=...   # only for a non-OpenAI endpoint

import zeroproof.simulations as zps

data = zps.simulate(agent="openai:gpt-4.1-mini", tools=my_tools,
                    system_prompt=my_system_prompt, output="rollout.jsonl")

Or use ZeroProof-hosted Qwen, which is the default when no agent= is given. Ask us for a VLLM_API_KEY; the endpoint is shared and rate limited.

export VLLM_API_KEY=...

Working in this repo: uv sync, then uv run pytest after uv sync --extra dev.

One runtime dependency (requests), Python 3.10+. Installing from PyPI rather than a path or a git URL matters if you build a Prime Intellect environment on this: the Environments Hub installs a pushed env with plain pip, so a [tool.uv.sources] git pin resolves locally and then fails on their runtime with a ModuleNotFoundError.

import zeroproof.simulations as zps

data = zps.simulate(tools=my_tools, system_prompt=my_system_prompt, output="rollout.jsonl")
data = zps.simulate(agent=my_agent)

Pass spec= if you have a local tools-and-system-prompt folder. The generated datasets are on Hugging Face, organized by agent type instead of stored in this repo.

What to run

Depends on the use case. How each scenario is built is in The recipe.

zps.simulate(tools=my_tools, system_prompt=my_system_prompt)                 # explore
zps.simulate(tools=my_tools, system_prompt=my_system_prompt, mode="sft")
zps.simulate(tools=my_tools, system_prompt=my_system_prompt, mode="rl")
zps.simulate(tools=my_tools, system_prompt=my_system_prompt, mode="adaptive", until="saturation")

Examples

Store datasets on Zero Proof Labs

Push a run to your Zero Proof Labs account so the optimization framework can iterate on it. For runtime SDK access, prefer a short-lived delegated credential (zp_dc_...) issued from a valid Clerk session token. The SDK uses ZEROPROOF_DELEGATED_CREDENTIAL by default; the legacy ZEROPROOF_API_KEY still works for compatibility.

# Preferred runtime path
# export ZEROPROOF_DELEGATED_CREDENTIAL="zp_dc_..."

# If you need to mint one from a Clerk session token:
# credential = zps.issue_delegated_credential(clerk_token, ttl_seconds=3600)
# export ZEROPROOF_DELEGATED_CREDENTIAL=credential["credential"]

data = zps.simulate(spec="specs/github")
v1 = data.push("github-explore-v1")            # -> {"datasetId": "ds_...", ...}

# iterate, then push the next version with lineage
v2 = data.push("github-explore-v2", parent=v1["datasetId"])

zps.datasets()                                  # list yours + storage used
rows = zps.pull(v1["datasetId"])               # rows, or pass path= for a file
zps.push_file("rollout.jsonl")                 # upload an existing JSONL
zps.delete_dataset(v1["datasetId"])            # permanent

Storage is private per account, 5 GB free. parent= records dataset lineage so iterations show as a family on the platform.

Speed

Two-minute airline runs using ZeroProof-hosted Qwen. Results were measured on the hosted GPU with warm replicas and burst under load.

Parameter reference

Aliases: phrasings= / n= → requests_per_situation; repeats= → rollouts_per_request; unique= → unique_situations; policy= → system_prompt; risk= → fault_rate.

Output

Each row: prompt, messages, steps, final_text, scenario_id. Optional world_state, faults, reward, reason. llm_grade=True adds llm_reward. zps.rank(path) adds quality without changing reward.

Every row carries schema_version ("1"). A row is a projection of four objects in zeroproof.simulations.schema: Task (the situation), Rollout (one episode), Judgment (a scorer's verdict), Marker (a behavior measurement). zps.from_row(row) splits a row into them and zps.to_row(...) flattens them back. The wire contract is zeroproof/simulations/schemas/row-v1.json. Rows written before the stamp are version 0 and load by shape, so older files still work.

The recipe

Each scenario is a draw across the world and the human.

World (from this agent's tools and system prompt)

• objects and tool results that match the spec
• tool outcome: success, timeout, deny, stale, etc.
• world state: exists, missing, already handled, unfinished, etc.
• history: first visit, prior miss, return, etc.
• rules the agent is supposed to follow

Human

• intent: which tool, what they want (randomized sometimes)
• stance: ordinary, ambiguous, adversarial, hurried, etc.
• persona: first time, returning, in a hurry, etc.
• tone: impatient, frustrated, polite, etc.
• typing: standard, lowercase, typo, clipped, etc.

Ordinary asks first, then the edges. On top of that, we embed the openers and add a bit of random noise so the batch stays spread out, not a cluster of near-copies. Spend the row cap and the clock on diversity, not copies.

Package layout

The public surface is the package itself: import zeroproof.simulations as zps. Internals are grouped by stage and may move between releases.

License

Apache-2.0