traigent 0.13.0.dev1

Enterprise-grade LLM optimization platform with advanced analytics and AI-powered insights

Traigent

Traigent is an AI Agent infrastructure that allows companies to take AI agents out of the lab and deploy them at high scale with high confidence.

Our mission: Anything you can measure, we can improve. Whether it's accuracy, speed of response, cost, or any other business metric — we bring strong results that deliver real business value.

Runs multiple LLM trials — run python -m traigent.examples.quickstart for a no-cost demo, or call enable_mock_mode_for_quickstart() from traigent.testing in local tutorial code. Set TRAIGENT_RUN_COST_LIMIT=2.0 as a best-effort local guardrail and set hard caps with your LLM/cloud providers; actual billing is determined by those providers. The legacy TRAIGENT_MOCK_LLM=true env var remains for backwards compatibility and is disabled when ENVIRONMENT=production. See Cost Management.

Quick Install:

macOS / Linux:

git clone https://github.com/Traigent/Traigent.git
cd Traigent

python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[recommended]"

Windows PowerShell:

git clone https://github.com/Traigent/Traigent.git
cd Traigent

python -m venv .venv
.venv\Scripts\Activate.ps1
pip install -e ".[recommended]"

For more options, see Installation details.

Try it now - no API keys needed:

pip install "traigent[integrations]"
python -m traigent.examples.quickstart

Or from a source checkout:

python hello_world.py

Here's what the quickstart does - one decorator, automatic optimization:

from langchain_openai import ChatOpenAI
import traigent

@traigent.optimize(
    configuration_space={
        "model": ["gpt-4o-mini", "gpt-4o"],
        "temperature": [0.0, 0.7, 1.0],
    },
    objectives=["accuracy"],
    eval_dataset="qa_samples.jsonl",
)
def answer(question: str) -> str:
    cfg = traigent.get_config()
    llm = ChatOpenAI(model=cfg["model"], temperature=cfg["temperature"])
    return llm.invoke(question).content

---

Using it in your own code

Add @traigent.optimize() to any function that calls an LLM — no framework required:

import traigent
import litellm                    # or openai, anthropic, requests …

@traigent.optimize(
    configuration_space={
        "model": ["gpt-4o-mini", "gpt-4o"],
        "temperature": [0.0, 0.7, 1.0],
    },
    objectives=["accuracy"],
    eval_dataset="path/to/your_evals.jsonl",
)
def your_function(question: str) -> str:
    cfg = traigent.get_config()
    response = litellm.completion(
        model=cfg["model"],
        temperature=cfg["temperature"],
        messages=[{"role": "user", "content": question}],
    )
    return response.choices[0].message.content

Works with any LLM provider — OpenAI, Anthropic, LiteLLM (100+ providers), or plain HTTP calls.

Portal · Quickstart · Examples · Skill · Walkthrough

---

Choose Your Path

Goal	Resource	Time
Get started quickly	Quick Start Guide	5 min
Understand the architecture	Architecture Overview	5 min
Track local runs in the portal	Hybrid portal tracking	5 min
Try examples locally, then make runs portal-visible	Mock walkthrough (8 steps) → Portal	15 min
Read the full API reference	Decorator Reference →	—

Full documentation index

Get started	Installation · 5-minute tutorial
User guides	Injection Modes · Configuration Spaces · Evaluation
Tunable Variable Language	TVL Guide
Advanced	Agent Optimization · Optuna Integration
API reference	Decorator Reference · Constraint DSL

---

🎬 See Traigent in Action — click to play demos

Demo
LLM Agent Optimization
Optimization Callbacks
Agent Configuration Hooks

🏗️ Architecture Overview — how it works

1. Suggest — the optimizer proposes a configuration to test
2. Inject — Traigent overrides your function's parameters with the proposed config
3. Evaluate — your function runs against the dataset, scored by the evaluator
4. Record — results update the optimizer's model
5. Repeat — loop continues until budget/trials exhausted, then outputs results

Read the full architecture guide →

---

🚀 Walkthrough — 8 runnable examples

The walkthrough examples use local mock mode through the quickstart/testing helpers with cost approval pre-set for dry runs — no provider API keys needed when calls go through LiteLLM or LangChain.

Show all 8 walkthrough steps

#	Run	What you'll learn
1	python walkthrough/mock/01_tuning_qa.py	Basic model + temperature optimization
2	python walkthrough/mock/02_zero_code_change.py	Seamless mode — zero code changes to existing code
3	python walkthrough/mock/03_parameter_mode.py	Explicit config access via traigent.get_config()
4	python walkthrough/mock/04_multi_objective.py	Balance accuracy, cost, and latency
5	python walkthrough/mock/05_rag_parallel.py	RAG optimization with parallel evaluation
6	python walkthrough/mock/06_custom_evaluator.py	Define your own success metrics
7	python walkthrough/mock/07_multi_provider.py	Compare OpenAI, Anthropic, Google in one run
8	python walkthrough/mock/08_privacy_modes.py	Local-only privacy-first execution

Browse reference examples → · Injection modes →

---

☁️ Traigent Portal & Hybrid Tracking

Connect to Traigent Portal to view results, compare trials, and collaborate. The supported portal-visible SDK path today is execution_mode="hybrid": trials run locally, while session and trial metrics are stored in the backend for portal tracking.

execution_mode="cloud" is reserved for future remote agent execution. It is not available yet and fails with: “Cloud remote execution is not available yet; use hybrid for portal-tracked optimization.”

1. Sign up at portal.traigent.ai — verify your email to activate
2. Create an API key — click your name (top-right) → API Keys → + Create API Key
3. Connect — run traigent auth login or set export TRAIGENT_API_KEY="sk_..."
4. Run with hybrid tracking — set execution_mode="hybrid" for portal-visible optimization

Credential priority and multi-provider setup

Credential	1st (highest)	2nd	3rd (default)
API Key	TRAIGENT_API_KEY env var	Stored CLI credentials	None (local only)
Backend URL	TRAIGENT_BACKEND_URL env var	Stored CLI credentials	portal.traigent.ai

Tip: No env vars needed after traigent auth login — the SDK picks up stored credentials automatically.

Multi-provider optimization — use LiteLLM to compare OpenAI, Anthropic, Google, Mistral, and 100+ providers:

@traigent.optimize(
    configuration_space={
        "model": ["gpt-4o-mini", "claude-3-haiku-20240307", "gemini/gemini-pro"],
        "temperature": [0.1, 0.5, 0.9],
    },
    objectives=["accuracy", "cost"],
    eval_dataset="data/qa_samples.jsonl",
)
def multi_provider_agent(question: str) -> str:
    config = traigent.get_config()
    response = litellm.completion(
        model=config.get("model"),
        temperature=config.get("temperature"),
        messages=[{"role": "user", "content": question}],
    )
    return response.choices[0].message.content

---

✨ Key Features

Feature	Description
Zero-code integration	Add @traigent.optimize() to existing code — no refactoring
Multi-algorithm	Random, Grid, Bayesian (TPE, NSGA-II, CMA-ES) via Optuna
Multi-objective	Optimize accuracy, latency, cost, and custom metrics simultaneously
Framework support	LangChain, OpenAI SDK, Anthropic, LiteLLM, and any LLM provider
Cost tracking	Integrated tokencost library with 500+ model pricing
Parallel execution	Concurrent trials and example-level parallelism
Error resilience	Interactive pause on rate limits and budget caps — resume or stop gracefully
Live progress	Auto-enabled progress bar in interactive terminals (progress_bar=False to disable)
Privacy-first	Local execution mode keeps all data on your machine

TraigentDemo — Streamlit playground, use cases, and research benchmarks

---

📦 Installation details, execution modes, CLI, and more

Installation

Python 3.11+ on Linux, macOS, or Windows. For coordinated release validation, install from this repository source tree.

Feature Set	Description
[recommended]	All user-facing features (default)
[integrations]	LangChain, OpenAI, Anthropic adapters
[analytics]	Visualization and analytics
[bayesian]	Bayesian optimization (TPE, NSGA-II)
[all]	Everything

Full installation guide →

Source install with pip:

git clone https://github.com/Traigent/Traigent.git
cd Traigent
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[recommended]"

Cost Management

Setting	How
Testing (no API calls)	Import and call enable_mock_mode_for_quickstart() from traigent.testing, or run python -m traigent.examples.quickstart for the demo
Cost Limit	TRAIGENT_RUN_COST_LIMIT=2.0 (default: $2/run)

Cost estimates, budgets, limits, alerts, and thresholds are best-effort software controls, not provider-side billing guarantees. Actual billing is determined by your LLM/cloud providers, and you remain responsible for provider charges. The legacy TRAIGENT_MOCK_LLM=true env var is supported only for backwards-compatible local scripts and is disabled when ENVIRONMENT=production. Mock mode skips the optimized-function pricing preflight for supported calls made through Traigent's integration/interceptor path; direct provider calls made outside that path should be stubbed explicitly for a guaranteed $0 rehearsal. See DISCLAIMER.md for details.

Evaluation

Provide a JSONL dataset — Traigent scores outputs using semantic similarity by default:

{"input": {"question": "What is AI?"}, "output": "Artificial Intelligence"}
{"input": {"question": "Explain ML"}, "output": "Machine learning uses data and algorithms"}

• input (required): your function's parameter names as keys
• output (optional): expected output for accuracy scoring

Evaluation guide → — custom evaluators, dataset formats, troubleshooting

Execution Modes

Mode	Status	Privacy	Algorithm	Best For
Local (edge_analytics)	✅ Available	✅ Complete	All (Random/Grid/Bayesian/Optuna)	Local/private runs
Hybrid (hybrid)	✅ Available	✅ Trial execution local	All (Random/Grid/Bayesian/Optuna)	Portal-tracked runs
Cloud (cloud)	🚧 Reserved	Not available	Future remote execution	Do not use yet

Execution modes guide → — mode comparisons, privacy details, migration path

Quick Reference

Parameter	Where	Description
configuration_space	@traigent.optimize()	Parameters to test (required)
objectives	@traigent.optimize()	Metrics to optimize for
eval_dataset	@traigent.optimize()	Dataset for evaluation
algorithm	.optimize() call	"random", "grid", "bayesian"
max_trials	.optimize() call	Number of configurations to test
progress_bar	.optimize() call	True / False / None (auto) — live progress bar

Injection Modes

Mode	Best for	How
Context (default)	Most cases	Inside the decorated function, call traigent.get_config() to read the active configuration (config.get("key", default)).
Seamless	Existing functions with simple local config defaults	Pass injection_mode="seamless"; Traigent rewrites simple local variable assignments that match configuration keys.
Parameter	New development	Pass injection_mode="parameter"; the decorated function receives a config argument with explicit config.get("key") access.

Injection modes guide →

CLI

traigent optimize module.py -a grid -n 10   # Run optimization
traigent validate data.jsonl -o accuracy     # Validate dataset
traigent results                             # List past runs
traigent plot <name> -p progress             # Visualize results
traigent auth login                          # Authenticate with portal
traigent --help                              # Full command reference

Troubleshooting

Problem	Fix
ModuleNotFoundError	pip install -e ".[recommended]" or check venv is activated
0.0% accuracy	Check dataset format; for local demos, import and call enable_mock_mode_for_quickstart() from traigent.testing
Missing API keys	Copy .env.example to .env; or run python -m traigent.examples.quickstart for a no-key demo
pytest rejects -n / --dist	Install dev test tooling first: pip install -e ".[all,dev]"
execution={"runtime": "node"} fails	Python SDK 0.12.0 removed the temporary JS bridge. Use native @traigent/sdk; see JS bridge migration.
Permission errors	Create a fresh venv and reinstall dependencies

---

🛠️ Development

python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[all,dev]"              # Install with dev dependencies
pytest                                   # Run tests
make format && make lint                 # Format and lint

Architecture guide → · Project structure →

🤝 Contributing

We welcome bug reports and feature requests via GitHub Issues. For security vulnerabilities, please email [e-mail].

📄 License

This project is dual-licensed: the GNU Affero General Public License v3.0 only (AGPL-3.0-only) - see LICENSE - or a Traigent commercial license under a separate written agreement (see COMMERCIAL-LICENSE.md). SPDX: AGPL-3.0-only OR LicenseRef-Traigent-Commercial. Commercial inquiries: [e-mail].

---

Get Started → | Examples → | Portal → | Skill → | Walkthrough → | GitHub Issues | Discussions