synth-containers

The task contract for Synth optimizers and evals — wrap any task as a small HTTP service that optimizers can target without touching your code.

PyPI · Optimizers · Cookbooks

A Synth container is a small HTTP service around a task. It owns the dataset, the scoring/verifier logic, the mutable prompt or policy fields, and the policy-model credential boundary. An optimizer like synth-optimizers GEPA sees one URL and a typed rollout contract — it never imports your task package or reads private evaluator state. The same contract works whether the task is a classifier, a coding agent, or a live game environment, and in Python, Rust, or TypeScript.

Install

pip install synth-containers
# or
uv add synth-containers

Local Synth development

Register the current checkout once after changing Containers or its package version:

./scripts/register-local-dev-build.sh

The command builds a wheel, installs it into an immutable machine-local directory, verifies the installed version, and atomically selects it for local Workshop builds. No launch flags or environment variables are required. Re-running it reuses an identical registered wheel.

Sibling Optimizers development remains editable through its checked-in uv source:

cd ../optimizers
uv sync --group dev

The contract

The Python SDK also provides Container, Container.serve(), ContainerHandle, ContainerConnection, and ContainerRunner (url, command, in-process app, or local image_id). CLI: synth-containers serve and synth-containers up.

Runnable local example (no provider credentials)

After pip install synth-containers, save this as smoke.py and run python smoke.py. It starts a loopback-only server, scores a deterministic exact-match task, prints reward: 1.0, and shuts the server down. This is an SDK/HTTP smoke test, not a model-quality evaluation.

from datetime import datetime, timezone
import httpx
from synth_containers import Container

container = Container("local-exact-match", default_submission_mode="sync")

@container.rollout
def rollout(payload):
    now = datetime.now(timezone.utc).isoformat()
    reward = float(payload["answer"] == payload["expected"])
    return {
        "rollout_id": payload["rollout_id"],
        "status": "completed", "success_status": "success",
        "created_at": now, "updated_at": now,
        "summary": {"reward": reward},
    }

with container.serve() as running:
    with httpx.Client(timeout=10, trust_env=False) as client:
        response = client.post(running.url + "/rollouts", json={
            "rollout_id": "smoke-1", "answer": "hello", "expected": "hello",
        })
        response.raise_for_status()
        result = response.json()
        assert result["status"] == "completed"
        assert result["summary"]["reward"] == 1.0
        print("reward:", result["summary"]["reward"])

See the cookbooks for complete containers: Banking77, HotpotQA, MiniGrid, TBLite, and Crafter.

Links

• Optimizers — GEPA on this contract
• Cookbooks — runnable containers
• Agent skill — drop into a coding agent to build and debug containers
• Contract OpenAPI

License

MIT