traigent 0.18.1.dev6

Enterprise-grade LLM optimization platform with advanced analytics and AI-powered insights

Traigent

Traigent is an AI Agent infrastructure that allows companies to take AI agents out of the lab and deploy them at high scale with high confidence.

Our mission: Anything you can measure, we can improve. Whether it's accuracy, speed of response, cost, or any other business metric — we bring strong results that deliver real business value.

Runs multiple LLM trials — run python -m traigent.examples.quickstart for a no-cost demo, or call enable_mock_mode_for_quickstart() from traigent.testing in local tutorial code. Set TRAIGENT_RUN_COST_LIMIT=2.0 as a best-effort local guardrail and set hard caps with your LLM/cloud providers; actual billing is determined by those providers. The legacy TRAIGENT_MOCK_LLM=true env var remains for backwards compatibility and is disabled when ENVIRONMENT=production. See Cost Management.

Quick Install:

Install from PyPI — no clone required (Python 3.11+):

# pip
pip install "traigent[recommended]"

# uv (into the active venv, or `uv add` into a uv project)
uv pip install "traigent[recommended]"
uv add "traigent[recommended]"

Prefer an isolated virtual environment first?

macOS / Linux:

python3 -m venv .venv
source .venv/bin/activate
pip install "traigent[recommended]"

Windows PowerShell:

python -m venv .venv
.venv\Scripts\Activate.ps1
pip install "traigent[recommended]"

For more options, see Installation details.

Try it now - no API keys needed:

pip install "traigent[integrations]"
python -m traigent.examples.quickstart

Or from a source checkout:

python hello_world.py

Here's what the quickstart does - one decorator, automatic optimization:

from langchain_openai import ChatOpenAI
import traigent

@traigent.optimize(
    configuration_space={
        "model": ["gpt-4o-mini", "gpt-4o"],
        "temperature": [0.0, 0.7, 1.0],
    },
    objectives=["accuracy"],
    eval_dataset="qa_samples.jsonl",
)
def answer(question: str) -> str:
    cfg = traigent.get_config()
    llm = ChatOpenAI(model=cfg["model"], temperature=cfg["temperature"])
    return llm.invoke(question).content

---

Using it in your own code

Add @traigent.optimize() to any function that calls an LLM — no framework required:

import traigent
import litellm                    # or openai, anthropic, requests …

@traigent.optimize(
    configuration_space={
        "model": ["gpt-4o-mini", "gpt-4o"],
        "temperature": [0.0, 0.7, 1.0],
    },
    objectives=["accuracy"],
    eval_dataset="path/to/your_evals.jsonl",
)
def your_function(question: str) -> str:
    cfg = traigent.get_config()
    response = litellm.completion(
        model=cfg["model"],
        temperature=cfg["temperature"],
        messages=[{"role": "user", "content": question}],
    )
    return response.choices[0].message.content

Works with any LLM provider — OpenAI, Anthropic, LiteLLM (100+ providers), or plain HTTP calls.

Portal · Quickstart · Examples · Skill · Walkthrough

---

Choose Your Path

Goal	Resource	Time
Get started quickly	Quick Start Guide	5 min
Understand the architecture	Architecture Overview	5 min
Track runs in the portal	Portal tracking	5 min
Try examples locally, then make runs portal-visible	Mock walkthrough (8 steps) → Portal	15 min
Read the full API reference	Decorator Reference →	—

Full documentation index

Get started	Installation · 5-minute tutorial
User guides	Injection Modes · Configuration Spaces · Evaluation
Tunable Variable Language	TVL Guide
Advanced	Agent Optimization
API reference	Decorator Reference · Constraint DSL

---

🎬 See Traigent in Action — click to play demos

Demo
LLM Agent Optimization
Optimization Callbacks
Agent Configuration Hooks

🏗️ Architecture Overview — how it works

1. Suggest — the optimizer proposes a configuration to test
2. Inject — Traigent overrides your function's parameters with the proposed config
3. Evaluate — your function runs against the dataset, scored by the evaluator
4. Record — results update the optimizer's model
5. Repeat — loop continues until budget/trials exhausted, then outputs results

Read the full architecture guide →

---

🚀 Walkthrough — 8 runnable examples

The walkthrough examples use local mock mode through the quickstart/testing helpers with cost approval pre-set for dry runs — no provider API keys needed when calls go through LiteLLM or LangChain.

Show all 8 walkthrough steps

#	Run	What you'll learn
1	python walkthrough/mock/01_tuning_qa.py	Basic model + temperature optimization
2	python walkthrough/mock/02_zero_code_change.py	Seamless mode — zero code changes to existing code
3	python walkthrough/mock/03_parameter_mode.py	Explicit config access via traigent.get_config()
4	python walkthrough/mock/04_multi_objective.py	Balance accuracy, cost, and latency
5	python walkthrough/mock/05_rag_parallel.py	RAG optimization with parallel evaluation
6	python walkthrough/mock/06_custom_evaluator.py	Define your own success metrics
7	python walkthrough/mock/07_multi_provider.py	Compare OpenAI, Anthropic, Google in one run
8	python walkthrough/mock/08_privacy_modes.py	No-egress local execution

Browse reference examples → · Injection modes →

---

☁️ Traigent Portal Tracking

Connect to Traigent Portal to view results, compare trials, and collaborate. Portal tracking is enabled automatically when TRAIGENT_API_KEY is set; most users do not need any routing settings.

The default run uses Traigent's smart optimizer when portal credentials are available. Local grid and random searches still sync results to the portal when authenticated. Use offline=True only when a run must avoid Traigent backend egress; offline runs do not sync.

1. Sign up at portal.traigent.ai — verify your email to activate
2. Create an API key — click your name (top-right) → API Keys → + Create API Key
3. Connect — run traigent auth login or set export TRAIGENT_API_KEY=”sk_...”
4. Run — portal tracking is automatic

Credential priority and multi-provider setup

Credential	1st (highest)	2nd	3rd (default)
API Key	TRAIGENT_API_KEY env var	Stored CLI credentials	None (local only)
Backend URL	TRAIGENT_BACKEND_URL env var	Stored CLI credentials	portal.traigent.ai

Tip: No env vars needed after traigent auth login — the SDK picks up stored credentials automatically.

Multi-provider optimization — use LiteLLM to compare OpenAI, Anthropic, Google, Mistral, and 100+ providers:

@traigent.optimize(
    configuration_space={
        "model": ["gpt-4o-mini", "claude-haiku-4-5-20251001", "gemini/gemini-pro"],
        "temperature": [0.1, 0.5, 0.9],
    },
    objectives=["accuracy", "cost"],
    eval_dataset="data/qa_samples.jsonl",
)
def multi_provider_agent(question: str) -> str:
    config = traigent.get_config()
    response = litellm.completion(
        model=config.get("model"),
        temperature=config.get("temperature"),
        messages=[{"role": "user", "content": question}],
    )
    return response.choices[0].message.content

---

✨ Key Features

Feature	Description
Zero-code integration	Add @traigent.optimize() to existing code — no refactoring
Multi-algorithm	Random and Grid locally; Bayesian (TPE, NSGA-II, CMA-ES) via the Traigent cloud
Multi-objective	Optimize accuracy, latency, cost, and custom metrics simultaneously
Framework support	LangChain, OpenAI SDK, Anthropic, LiteLLM, and any LLM provider
Cost tracking	Integrated tokencost library with 500+ model pricing
Parallel execution	Concurrent trials and example-level parallelism
Error resilience	Interactive pause on rate limits and budget caps — resume or stop gracefully
Live progress	Auto-enabled progress bar in interactive terminals (progress_bar=False to disable)
No-egress option	offline=True keeps Traigent optimization traffic local when policy requires it

TraigentDemo — Streamlit playground, use cases, and research benchmarks

---

📦 Installation details, optimization routing, CLI, and more

Installation

Python 3.11+ on Linux, macOS, or Windows. The published package on PyPI is the recommended way to install — pip install "traigent[recommended]" or uv add "traigent[recommended]". No repository checkout is required to use the SDK or its traigent CLI.

Feature Set	Description
[recommended]	All user-facing features (default)
[integrations]	LangChain, OpenAI, Anthropic adapters
[analytics]	Visualization and analytics
[bayesian]	Bayesian optimization dependencies (used by the Traigent cloud; not available locally)
[all]	Everything

Full installation guide →

Contributor / source install (only needed to hack on Traigent itself or run the coordinated release-validation suite — not required to use the SDK):

git clone https://github.com/Traigent/Traigent.git
cd Traigent
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[recommended]"

Cost Management

Setting	How
Testing (no API calls)	Import and call enable_mock_mode_for_quickstart() from traigent.testing, or run python -m traigent.examples.quickstart for the demo
Cost Limit	TRAIGENT_RUN_COST_LIMIT=2.0 (default: $2/run)

Cost estimates, budgets, limits, alerts, and thresholds are best-effort software controls, not provider-side billing guarantees. Actual billing is determined by your LLM/cloud providers, and you remain responsible for provider charges. The legacy TRAIGENT_MOCK_LLM=true env var is supported only for backwards-compatible local scripts and is disabled when ENVIRONMENT=production. Mock mode skips the optimized-function pricing preflight for supported calls made through Traigent's integration/interceptor path; direct provider calls made outside that path should be stubbed explicitly for a guaranteed $0 rehearsal. See DISCLAIMER.md for details.

Evaluation

Provide a JSONL dataset — Traigent scores outputs by exact / case-insensitive match by default. For semantic scoring (paraphrases, summarization, translation), supply a scoring_function (embeddings or LLM-judge) or a custom_evaluator:

{"input": {"question": "What is AI?"}, "output": "Artificial Intelligence"}
{"input": {"question": "Explain ML"}, "output": "Machine learning uses data and algorithms"}

• input (required): your function's parameter names as keys
• output (optional): expected output for accuracy scoring

Evaluation guide → — custom evaluators, dataset formats, troubleshooting

Optimization Routing

Request	Where optimization decisions come from	Portal sync
Omit routing settings	Traigent smart optimizer when authenticated	Yes
algorithm="grid" or "random"	Local search in your Python process	Yes, unless offline=True
Smart algorithm name, such as "bayesian" or "optuna_tpe"	Traigent cloud optimizer	Yes
offline=True	Local only, zero Traigent backend egress	No

Most users should omit these settings. Use grid or random for explicit local search; use offline=True only for no-egress runs.

Optimization routing guide → — defaults, local search, portal sync, and migration notes

Quick Reference

Parameter	Where	Description
configuration_space	@traigent.optimize()	Parameters to test (required)
objectives	@traigent.optimize()	Metrics to optimize for
eval_dataset	@traigent.optimize()	Dataset for evaluation
algorithm	.optimize() call	"random", "grid" (local); smart algorithms run in the Traigent cloud
max_trials	.optimize() call	Number of configurations to test
progress_bar	.optimize() call	True / False / None (auto) — live progress bar

Injection Modes

Mode	Best for	How
Context (default)	Most cases	Inside the decorated function, call traigent.get_config() to read the active configuration (config.get("key", default)).
Seamless	Existing functions with simple local config defaults	Pass injection_mode="seamless"; Traigent rewrites simple local variable assignments that match configuration keys.
Parameter	New development	Pass injection_mode="parameter"; the decorated function receives a config argument with explicit config.get("key") access.

Injection modes guide →

CLI

traigent optimize module.py -a grid -n 10   # Run optimization
traigent validate data.jsonl -o accuracy     # Validate dataset
traigent results                             # List past runs
traigent plot <name> -p progress             # Visualize results
traigent auth login                          # Authenticate with portal
traigent --help                              # Full command reference

Troubleshooting

Problem	Fix
ModuleNotFoundError	pip install -e ".[recommended]" or check venv is activated
0.0% accuracy	Check dataset format; for local demos, import and call enable_mock_mode_for_quickstart() from traigent.testing
Missing API keys	Copy .env.example to .env; or run python -m traigent.examples.quickstart for a no-key demo
pytest rejects -n / --dist	Install dev test tooling first: pip install -e ".[all,dev]"
execution={"runtime": "node"} fails	Python SDK 0.12.0 removed the temporary JS bridge. Use native @traigent/sdk; see JS bridge migration.
Permission errors	Create a fresh venv and reinstall dependencies

---

🛠️ Development

python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[all,dev]"              # Install with dev dependencies
pytest                                   # Run tests
make format && make lint                 # Format and lint

Architecture guide → · Project structure →

🤝 Contributing

We welcome bug reports and feature requests via GitHub Issues. For security vulnerabilities, please email [e-mail].

📄 License

This project is dual-licensed: the GNU Affero General Public License v3.0 only (AGPL-3.0-only) - see LICENSE - or a Traigent commercial license under a separate written agreement (see COMMERCIAL-LICENSE.md). SPDX: AGPL-3.0-only OR LicenseRef-Traigent-Commercial. Commercial inquiries: [e-mail].

---

Get Started → | Examples → | Portal → | Skill → | Walkthrough → | GitHub Issues | Discussions