tei-loop 0.1.5

Target, Evaluate, Improve: A self-improving loop for agentic systems

TEI Loop

Target, Evaluate, Improve — a self-improving loop for agentic systems.

Get Started (2 steps)

pip install tei-loop

tei your_agent.py

That's it. TEI auto-detects your agent function, generates a test query, evaluates across 4 dimensions, and applies targeted improvements.

If pip is not found, use pip3. If tei is not found, use python3 -m tei_loop.cli.

What Happens When You Run tei your_agent.py

1. TEI loads your file and finds the agent function (any callable that takes input and returns output)
2. Auto-generates a relevant test query based on your agent's purpose
3. Runs your agent and evaluates the output across 4 dimensions
4. If any dimension can be improved, applies a targeted fix and re-runs
5. Prints before/after scores

Options

python3 -m tei_loop agent.py                                    # Full loop, auto-detected
python3 -m tei_loop agent.py --function my_summarizer            # Specify which function
python3 -m tei_loop agent.py --query "custom input"              # Custom test query
python3 -m tei_loop agent.py --mode compare                      # Before/after comparison
python3 -m tei_loop agent.py --mode evaluate                     # Baseline only
python3 -m tei_loop agent.py --retries 5                         # More improvement cycles
python3 -m tei_loop agent.py --verbose                           # Detailed output

If your file has multiple functions, use --function to specify which one is your agent.

Python API

import asyncio
from tei_loop import TEILoop

def my_agent(query: str) -> str:
    # your agent logic
    return result

async def main():
    loop = TEILoop(agent=my_agent)
    result = await loop.run("your test query")
    print(result.summary())

asyncio.run(main())

4 Evaluation Dimensions

Dimension	What it checks
Target Alignment	Did the agent pursue the correct objective?
Reasoning Soundness	Was the reasoning logical and non-contradictory?
Execution Accuracy	Were the right tools called with correct parameters?
Output Integrity	Is the output complete, accurate, and consistent?

Two Modes

Runtime mode (default): Per-query, 1-3 retries, fixes individual failures in seconds.

Development mode: Across many queries, proposes permanent prompt improvements.

dev_results = await loop.develop(
    queries=["query1", "query2", "query3", ...],
    max_iterations=50,
)

How It Works

TEI wraps any Python callable as a black box. No code changes to your agent needed.

When a dimension fails, TEI applies the right fix strategy:

Failure	Fix Strategy
Target drift	Re-anchor to original objective
Flawed reasoning	Regenerate plan with failure context
Execution errors	Correct tool calls and parameters
Output issues	Repair factual errors and fill gaps

Configuration

TEI auto-detects your LLM provider from environment variables (OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY). No new accounts or API keys needed.

loop = TEILoop(
    agent=my_agent,
    eval_llm="gpt-5.2",       # Smartest for evaluation
    improve_llm="gpt-5-mini", # Cost-effective for fixes
)

Cost

Scenario	Cost
Agent passes all dimensions	~$0.05
One improvement cycle	~$0.10
Full 3-retry loop	~$0.25

Works With Any Agent

TEI wraps any Python callable. No framework lock-in:

• LangGraph agents
• CrewAI crews
• Custom Python functions
• FastAPI endpoints
• Any callable that takes input and returns output

License

MIT