💧 Evaporator

An AI application should need less AI over time.

Evaporator reads the logs of your high-volume LLM calls, writes ordinary code for the predictable part of that traffic, proves it on held-out data, and ships it to you as a pull request with a runtime guard. The model still handles everything the code isn't sure about.

Routers pick a cheaper model. Caches reuse old answers. Distillers train a smaller model.
Evaporator removes the model call.

Quickstart · Connect · How it works · Safety model · Runtime guard · Control plane · Coding agents · Status

_{A real, unedited run on a signed-in Claude Code (long model waits cut). Synthetic support-ticket logs: 120,000 calls a month, about a third of them messy or ambiguous.
Result: 73.1% of calls answered by code, 100% accurate on 24,000 held-out calls, 0 rare-class errors, generated tests passed; the ambiguous tickets stay on the LLM.
MP4 version · reproduce it with docs/demo}

✨ The whole product in one interaction

$ evaporator scan traces.jsonl --labels labels.csv
Loaded 20,000 calls from traces.jsonl across 1 call site(s); cost known for 100%, ground truth for 20,000.
Found 1 compilation candidate.

  classify_ticket()   70.1% validated offline

classify_ticket()
  Validated offline coverage         70.1%
  Held-out examples                  4,000
  Accuracy vs ground truth         100.00%
  LLM accuracy on same slice        98.61%
  Rare-class errors                      0
  Rules graduated                   4 of 6
  outcome preserved on held-out test

$ evaporator compile classify_ticket --target app/tickets.py:classify_ticket
✓ generated implementation      evaporated/classify_ticket.py
✓ generated abstention boundary evaporated/classify_ticket.envelope.json
✓ generated tests               tests/evaporated/test_classify_ticket.py (2,094 cases)
✓ 20,000 historical cases replayed: 14,052 answered by code (100.00% correct), 5,948 left to the LLM, 0 rare-class errors
✓ generated tests: 6 passed
✓ GitHub PR opened

_{Output from the bundled synthetic support-ticket dataset. Your numbers depend on your traffic, and finding them is the point of running scan.}

What you get is a pull request: ordinary source code you review, own and merge. The diff at your call site is one line:

from evaporated.classify_ticket import evaporate

@evaporate                      # 70% of calls answered by code, the rest go to the LLM exactly as before
def classify_ticket(text):
    return call_llm(text)

🧠 The core idea: compile + abstain

The code doesn't need to know everything. It needs to know two things very well: what it can answer, and when it must abstain. The second is the hard part, and it's what Evaporator is built around.

flowchart LR
    A[Input] --> B{Inside the validated<br/>envelope?}
    B -- yes --> C[Compiled code]
    B -- no --> D[Original LLM call]
    C --> E[Result]
    D --> E
    C -. 1% shadow sample .-> D

• Partial coverage is success. "68% runs as code, 32% still goes to the model" is the expected outcome, and abstaining isn't a failure.
• Measure the real outcome, not agreement. Code that copies the LLM's mistakes scores 99.99% agreement and proves nothing. When you supply ground truth, it's the bar. When you don't, the report says so, every time.
• Report only what the evidence supports. Rare or thinly observed regions stay on the LLM.
• You own the code. It's plain Python in your repo, with no black-box runtime.

🚀 Quickstart

uvx evaporator                  # try it without installing (needs uv: https://docs.astral.sh/uv/)

To keep it installed, pick one:

uv tool install evaporator      # recommended
pipx install evaporator
pip install evaporator          # into an existing Python 3.11+ environment
uv tool install "git+https://github.com/soummyaanon/evaporator"   # latest unreleased main

Run evaporator in your project folder. It opens a chat with the prompt ready, finds your call logs and your AI connection by itself and says in one line what it found. Then type:

which of my AI calls can become code?

You get each call site with its calls, its spend and a plain label: looks like it can become code (not proven yet), stays on the AI (open-ended) or too few calls. The chat then offers to prove the top candidate on held-out calls, naming the model it will use. A scan, a compile and a pull request each run only after you say yes.

Slash commands in the same prompt:

Prefer the commands, or working from a clone?

git clone https://github.com/soummyaanon/evaporator && cd evaporator
python3 -m venv .venv && source .venv/bin/activate
pip install -e '.[all]'

evaporator setup                                           # connect a model once (see Connect below)
evaporator scan your-export.jsonl --list                   # profile call sites; no model calls, no cost
evaporator scan your-export.jsonl --labels truth.csv       # synthesize + validate
evaporator compile <callsite> --target app/file.py:func    # branch, tests, PR

evaporator setup reruns the connection steps without opening the app. Without a terminal it prints how to configure through the environment instead.

In a terminal, scan and report draw the call site as a small isometric evaporator: the tank boils down to the share that still needs the LLM, and the steam that rises turns into code. Pipes and CI get plain text; set EVAPORATOR_PLAIN=1 to force it.

Supported log formats: OpenAI and Anthropic request/response logs, Langfuse exports, Evaporator's native JSONL, and any JSONL or CSV through a small TOML field mapping:

[fields]
id = "request_id"
timestamp = "created_at"
input = "payload.text"
output = "result.label"
ground_truth = "reviewed_label"

🗺 No logs yet? Map your code

Evaporator proves things on real traffic, so it needs a log of your LLM calls. If you don't have one, start the chat and type map my code:

1. The connected agent reads your code, in any language. That's your Claude Code, Codex or API-key model. It gets three read-only Evaporator tools: list, search and read files. They stay inside the project and never open .env files, keys, credentials, dependencies, build output or hidden folders.
2. You get a map: every LLM call, with its file and function, its model, whether its output looks bounded (a schema, label set or JSON mode) or open-ended, and whether it's already logged (Langfuse, LangSmith, Helicone, …).
3. Recording, in your repo's own language. For bounded calls that aren't logged yet, the agent writes a small recorder (Python, TypeScript or anything else) to Evaporator's log format and proposes the change. You see exactly which files change in an "Add recording to your code?" dialog. Only after your yes does Evaporator apply it, on a new branch (evaporator/record-calls). Nothing is pushed, and the recorder adds no dependency.

Deploy that branch, let real traffic flow for a while, then ask "which of my AI calls can become code?". Reading code finds candidates; only recorded traffic proves anything.

🔌 Connect

Synthesis needs a model. The chat picks one automatically: the one you chose before, else a signed-in Claude Code, else a signed-in Codex, else a saved API key, else ANTHROPIC_API_KEY / OPENAI_API_KEY. The header always names the connection in use; /model changes it (as does evaporator setup). If nothing is found, the chat asks you to connect one:

• Claude Code. Uses the claude you already signed in to. Evaporator runs the binary and never reads its token store.
• Codex. The same, through your codex binary.
• API key. Anthropic, OpenAI, or any OpenAI-compatible gateway.

Subscription usage counts against your Claude or ChatGPT plan. Evaporator writes no chat transcript, but when the chat runs on Claude Code or Codex those tools keep their own session history, as they do for any use.

Key precedence: --api-key-env / EVAPORATOR_API_KEY_ENV beats a saved credential, which beats ANTHROPIC_API_KEY / OPENAI_API_KEY. Provider and model: flag, then EVAPORATOR_* environment, then your saved config, then the default.

Where things live:

A key is never printed, logged or written anywhere else.

Log out. /logout in the chat, or evaporator logout in a terminal, forgets the chosen connection and model and, for an API key, deletes that saved key; evaporator logout --all deletes every saved key (--yes skips the question, for scripts). Both ask first and never print a key. Claude Code and Codex stay signed in for your own use; to sign out of them too, run claude auth logout or codex logout. A key still exported in your shell (ANTHROPIC_API_KEY / OPENAI_API_KEY) can still be found until you unset it, and the logout message says so.

Personal data. Before the first scan in a project, and before Claude Code or Codex can otherwise reach it (Codex's first chat turn, /agent, /dashboard), the chat asks once whether the data contains patient or personal data, and saves the answer as sensitive in project.toml. Subscriptions are not BAA-covered, so on a sensitive project the chat warns once before it uses claude-code or codex ("Use anyway" or "Connect an API key"). "Use anyway" is saved as allow_subscription = true in project.toml and the header then shows "personal data · not BAA-covered"; "Connect an API key" ends that conversation and asks for a key on a BAA-covered endpoint. Until the warning is accepted (or you set allow_subscription = true yourself), the CLI, MCP and the dashboard refuse subscriptions on that project. (Changed on 2026-10-18, decided by the user: a sensitive project used to refuse subscriptions outright.)

Nothing runs without your yes. A scan runs only after you agree to it in the chat, with the model named. Writing a compile into a repository needs a yes naming the repository, and pushing or opening a pull request needs a second, separate yes.

⚙️ How it works

flowchart TD
    A[Exported LLM logs] --> B[Ingest + group by call site]
    B --> C{Bounded output?}
    C -- open-ended --> L[Stays on the LLM]
    C -- labels or JSON fields --> D[Time-ordered split<br/>60 / 20 / 20]
    D --> E[Synthesize rules<br/>from de-identified examples]
    E --> F[Graduate each rule<br/>on validation evidence]
    F -->|refine| E
    F --> G[Score once on<br/>untouched test split]
    G --> H[Compile: code + envelope<br/>+ guard + tests]
    H --> I[Pull request]
    I --> J[Runtime guard<br/>shadow · drift · revert]
    J --> K[Control plane]

🛡 Safety model

🪝 The runtime guard

The guard is stdlib-only and gets copied into your repo inside the PR, so there's nothing new to install.

• Sync and async def, including methods: @evaporate(input_arg=1), with custom input extraction and output mapping.
• Shadow sampling. 1% of compiled answers are re-checked against the LLM in the background. Too many mismatches and the guard reverts itself.
• Revert without a deploy. EVAPORATOR_REVERT=classify_ticket (or *), a revert file, or one command through the control plane.
• Drift detection. It warns when the share of out-of-envelope inputs rises above what was seen offline.

📡 Control plane

evaporator serve --token "$TOKEN" --slack-webhook "$SLACK_URL"
export EVAPORATOR_TELEMETRY_URL=http://control:8787/v1/telemetry EVAPORATOR_TELEMETRY_TOKEN="$TOKEN"

evaporator status                                     # fleet view
evaporator revert classify_ticket --reason "incident" # every instance, at its next check-in
evaporator resume classify_ticket

┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━┓
┃ call site       ┃ instances ┃ calls ┃ compiled ┃ shadow agree ┃ outside envelope ┃ saved ┃ state ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━┩
│ classify_ticket │ 1/1       │ 100   │ 60.0%    │ n/a          │ 20.0%            │ $0.05 │ ok    │
└─────────────────┴───────────┴───────┴──────────┴──────────────┴──────────────────┴───────┴───────┘

It's a stdlib HTTP server with SQLite storage.

• Aggregation. Combines telemetry from every instance and ignores out-of-order snapshots.
• Alerts. Sends drift and revert alerts to Slack or email, at most once an hour per call site and kind.
• Auto-revert. Reverts a call site fleet-wide when combined shadow mismatch since the last resume crosses the envelope's threshold.
• New instances. A new instance picks up an active revert at startup.

🤖 Use it from a coding agent

evaporator mcp is a stdio MCP server, so fx, Claude Code, Cursor and other agents can ask Evaporator to do the work instead of hand-writing an unproven classifier. The agent gets four tools:

Bring your own model. Synthesis runs on your key, chosen when you start the server, never by the agent:

# Anthropic (default)
export ANTHROPIC_API_KEY=...

# OpenAI, or any OpenAI-compatible gateway
export OPENAI_API_KEY=...
export EVAPORATOR_PROVIDER=openai EVAPORATOR_MODEL=<model>   # EVAPORATOR_BASE_URL=<gateway> optional

The same EVAPORATOR_* variables work for evaporator scan. The agent can't change the model, the endpoint, the thresholds or de-identification, and it can't score its own rules against the test split. Those switches exist only on the CLI.

fx: add the server and the skill:

/mcp add evaporator evaporator mcp
/skills install soummyaanon/evaporator --skill evaporator

A scan takes minutes, longer than fx's default 60-second MCP timeout, so raise it in ~/.fx/mcp.json:

{
  "mcp": {
    "evaporator": {
      "type": "local",
      "command": ["evaporator", "mcp"],
      "environment": { "EVAPORATOR_PROVIDER": "openai", "EVAPORATOR_MODEL": "<model>" },
      "operation_timeout_ms": 1800000
    }
  }
}

Claude Code: claude mcp add evaporator -- evaporator mcp, and copy skills/evaporator into .claude/skills/.

Then ask: "Find the LLM calls in logs/traces.jsonl that should be code."

🧭 Where it fits

In scope: classification, routing, normalization, validation and structured extraction, meaning functions where a correct output can be stated and checked mechanically.
Out of scope: open-ended generation, conversational replies, multi-step agents, and judgment-heavy decisions.

🔐 Configuration reference

📍 Status

Pre-release. The CLI, guard and control plane are built and covered by 59 tests, including end-to-end scan → compile → run.

It has not yet been validated on real production traffic. The number that decides whether this is worth doing is what share of real inference dollars can be removed without degrading the measured outcome, and producing it is the first milestone:

• Scan, compile, PR, runtime guard, control plane
• Classification, routing and structured extraction
• Sandboxed synthesis and PHI de-identification
• Phase 0: measured results on three real call sites
• First design-partner PR merged, savings verified over 30 days
• TypeScript guard, GitLab/Bitbucket

🛠 Development

pip install -e '.[all,dev]'
pytest