This release is a pre-release and may not be stable for production use.
Synth
Serverless Posttraining APIs for Developers
Average accuracy on LangProBe prompt optimization benchmarks.
Demo Notebooks (Colab)
- GEPA Banking77 Prompt Optimization
- GEPA Crafter VLM Verifier Optimization
- GraphGen Image Style Matching
Highlights
- 🎯 GEPA Prompt Optimization - Automatically improve prompts with evolutionary search. See 70%→95% accuracy gains on Banking77, +62% on critical game achievements
- 🔍 Zero-Shot Verifiers - Fast, accurate rubric-based evaluation with configurable scoring criteria
- 🧬 GraphGen - Train custom verifier graphs optimized for your specific workflows. Train custom pipelines for other tasks
- 🚀 No Code Changes - Wrap existing code in a FastAPI app and optimize via HTTP. Works with any language or framework
- ⚡️ Local Development - Run experiments locally with tunneled task apps. No cloud setup required
- 🗂️ Multi-Experiment Management - Track and compare prompts/models across runs with built-in experiment queues
Getting Started
uv add
uv run synth-ai tui
OpenCode Skills (Synth API)
The Synth-AI TUI integrates with OpenCode and ships a synth-api skill.
# List packaged skills shipped with synth-ai
uvx synth-ai skill list
uvx synth-ai skill install synth-api --dir ~/custom/opencode/skill
Testing
Run the TUI integration tests:
cd synth_ai/_tui
bun test
Synth is maintained by devs behind the MIPROv2 prompt optimizer.
Documentation
Community
GEPA Prompt Optimization (SDK)
Run GEPA prompt optimization programmatically:
import asyncio
import os
from synth_ai.sdk.api.train.prompt_learning import PromptLearningJob
from synth_ai.sdk.localapi import LocalAPIConfig, create_local_api
# Create a local task app: app = create_local_api(LocalAPIConfig(app_id="my_app", handler=my_handler))
# Create and submit a GEPA job
pl_job = PromptLearningJob.from_dict({
"job_type": "prompt_learning",
"config": {
"prompt_learning": {
"gepa": {
"rollout": {"budget": 100},
"population_size": 10,
"generations": 5,
}
}
},
"task_app_id": "my_task_app",
})
pl_job.submit()
result = pl_job.stream_until_complete(timeout=3600.0)
print(f"Best score: {result.best_score}")
See the Banking77 demo notebook for a complete example with local task apps.
Zero-Shot Verifiers (SDK)
Run a built-in verifier graph with rubric criteria passed at runtime. See the Crafter VLM demo for verifier optimization:
import asyncio
import os
from synth_ai.sdk.graphs import VerifierClient
async def run_verifier():
client = VerifierClient(
base_url=os.environ["SYNTH_BACKEND_BASE"],
api_key=os.environ["SYNTH_API_KEY"],
)
result = await client.evaluate(
job_id="zero_shot_verifier_single",
trace={"session_id": "s", "session_time_steps": []},
rubric={
"event": [{"id": "accuracy", "weight": 1.0, "description": "Correctness"}],
"outcome": [{"id": "task_completion", "weight": 1.0, "description": "Completed task"}],
},
options={"event": True, "outcome": True, "model": "gpt-5-nano"},
policy_name="my_policy",
task_app_id="my_task",
)
return result
asyncio.run(run_verifier())
You can also call arbitrary graphs directly:
from synth_ai.sdk.graphs import GraphCompletionsClient
client = GraphCompletionsClient(base_url="https://api.usesynth.ai", api_key="...")
resp = await client.run(
graph={"kind": "zero_shot", "verifier_shape": "mapreduce", "verifier_mode": "rubric"},
input_data={"trace": {"session_id": "s", "session_time_steps": []}, "rubric": {"event": [], "outcome": []}},
)
GraphGen: Train Custom Verifier Graphs
Train custom verifier graphs using GraphGen. See the Image Style Matching demo for a complete GraphGen example:
from synth_ai.sdk.api.train.graphgen import GraphGenJob
# Train a verifier graph
verifier_job = GraphGenJob.from_dataset(
dataset="verifier_dataset.json",
graph_type="verifier",
policy_models=["gpt-4.1"],
proposer_effort="medium", # Use "medium" (gpt-4.1) or "high" (gpt-5.2)
rollout_budget=200,
)
verifier_job.submit()
result = verifier_job.stream_until_complete(timeout=3600.0)
# Run inference with trained verifier
verification = verifier_job.run_verifier(
trace=my_trace,
context={"rubric": my_rubric},
)
print(f"Score: {verification.score}, Reasoning: {verification.reasoning}")
1 maintainer
Metadata
1 maintainer
Release files for synth-ai 0.5.4.dev20260120
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| synth_ai-0.5.4.dev20260120.tar.gz | 1.3 MB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| synth_ai-0.5.4.dev20260120-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 3.2 MB
Release files / synth_ai-0.5.4.dev20260120.tar.gz
| Download URL | synth_ai-0.5.4.dev20260120.tar.gz |
|---|---|
| Size | 1.3 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ee38d368f451dae2d2fdbed86e1f988dbe8075de988e6fe580f421ece9a1313e
|
|
BLAKE2b-256 checksum How to use checksums |
26974a2856af29f9a6a16f2a31af7addd8a0858ba9bfb1cc3a643d93f12fb700
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.11
|
Release files / synth_ai-0.5.4.dev20260120-py3-none-any.whl
| Download URL | synth_ai-0.5.4.dev20260120-py3-none-any.whl |
|---|---|
| Size | 1.9 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
d250ca317fe3b0029b75fb4c5b8562922274e5294405d2b008aaed872dd06788
|
|
BLAKE2b-256 checksum How to use checksums |
3712d5ce881ab2a259e32aeae586ece934589d362040890a9ca4f9f9d0cd236c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.11
|
Security for synth-ai 0.5.4.dev20260120
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
