AgentFlight (agentflight)

AI Agent Black-Box Flight Recorder, Deterministic Trajectory Replay (<0.02 ms, $0.00 CI Spend), Differential Regression Detector, OpenTelemetry / W3C TraceContext Exporter (for stackpulse), Real-Time Loop/Cost Circuit Breaker & 10-Rule MCP Tool-Smell Linter.

🚀 Live Interactive Developer Portal & Playground: Visit https://agentflight-tau.vercel.app in your web browser to test tool linting live, inspect schemas, and copy 1-click configurations for Claude Code, Cursor, Windsurf, and Claude Desktop.

⭐ Like what you see? Give this repository a Star and follow creator Akandwanaho Ivan Alvin!

Why AgentFlight Exists (The 2026 Agent Reliability Pain Point)

Production AI agent engineering in 2026 faces two compounding bottlenecks:

1. Non-Deterministic, Expensive Multi-Step Agent Debugging & CI Testing: When a 15-step tool-calling or MCP agent fails on step 12, re-running live LLM turns and external API calls in CI is slow (10–45s), flaky, and expensive. Engineers need a "VCR / Black-Box Flight Recorder for AI Agents" that captures every LLM turn, MCP RPC, and tool invocation into a tamper-evident SHA-256 hash chain cassette and replays the entire trajectory deterministically in <0.02 ms with $0.00 token spend—raising a structural diff (TrajectoryDivergenceError) the instant an agent's behavior drifts from the golden baseline.
2. "MCP Tool Smell" (Schema Ergonomics Bugs): Empirical evaluations across Model Context Protocol (MCP) servers and function-calling suites show that most tool-selection failures are caused not by model reasoning limits, but by poorly authored tool schemas: missing parameter descriptions, ambiguous boolean flags (flag, mode), deeply nested input objects (depth > 3), unconstrained finite-set strings (status, format without enum), overlapping tool names (search_records vs. find_records), and destructive tools (delete_*, drop_*) without confirmation guards.

agentflight solves both problems with a pure-Python 3.10+ stdlib core engine (src/agentflight/ uses zero third-party runtime dependencies), paired with a FastAPI/ASGI portal gateway.

Architecture Blueprint

+-----------------------------------------------------------------------------------+
|                                   agentflight                                     |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  1. Pre-Deploy Static Gate                2. Runtime Execution & Guard            |
|  +----------------------------------+     +------------------------------------+  |
|  | ToolSmellLinter (linter.py)      |     | TrajectoryCircuitBreaker (guard.py)|  |
|  | - 10 Deterministic Rules         |     | - Exact Call Loop (N repeats)      |  |
|  |   (TS001 .. TS010)               |     | - Ping-Pong Cycle (A->B->A->B,     |  |
|  | - Agent Ergonomics Score (0-100) |     |   A->B->C->A->B->C)                |  |
|  | - Token Footprint Estimator      |     | - Consecutive Tool Error Storm     |  |
|  | - Actionable Remediations        |     | - Hard Token & USD Budget Caps     |  |
|  +-----------------+----------------+     +-----------------+------------------+  |
|                    |                                        |                     |
|                    v                                        v                     |
|  3. Tamper-Evident Flight Recorder        4. CI Replay & Golden Diff Engine       |
|  +----------------------------------+     +------------------------------------+  |
|  | FlightRecorder (recorder.py)     |     | CassetteReplayer & Evaluator       |  |
|  | - Append-only TrajectorySpan     | --> | - Sub-ms $0.00 Deterministic Stub  |  |
|  | - Canonical JSON SHA-256 Chain   |     | - Structural Diff on Divergence    |  |
|  |   (prev_hash -> step_hash)       |     | - Tool Precision / Recall / F1     |  |
|  | - .flight.json & .flight.jsonl   |     | - Step Efficiency & Cost Delta     |  |
|  | - verify_cassette() Tamper Check |     | - Goal Completion Score (0.0-1.0)  |  |
|  +----------------------------------+     +------------------------------------+  |
+-----------------------------------------------------------------------------------+

Installation

# Install from PyPI (module import & CLI command remain `agentflight`):
pip install agentflight-mcp

# Or install v1.3.1 directly from GitHub:
pip install "git+https://github.com/codesphere-u/agentflight.git@v1.3.1"

# Contributors:
pip install -e ".[dev]"

Quickstart

0. 1-Line Drop-In SDK & Pytest Integration (v1.3.0)

Wrap OpenAI, AsyncOpenAI, OpenRouter, vLLM, OpenAny, or Anthropic clients with zero boilerplate:

from openai import OpenAI
from agentflight import wrap_openai, wrap_anthropic, TrajectoryCircuitBreaker

client = wrap_openai(OpenAI(), guard=TrajectoryCircuitBreaker(max_cost_usd=0.25))
resp = client.chat.completions.create(model="gpt-4o-mini", messages=[{"role": "user", "content": "Health?"}])
client.agentflight_recorder.save("tests/cassettes/health.flight.json")

Pytest fixtures (flight_recorder, flight_guard, flight_session) and @pytest.mark.flight_cassette register automatically on pip install agentflight-mcp:

import pytest

@pytest.mark.flight_cassette("tests/cassettes/checkout.flight.json")
def test_checkout_deterministic(flight_session, flight_guard):
    search = flight_session.stub("search_catalog", fallback_fn=lambda q: {"hits": [q]})
    assert search(q="router")["hits"] == ["router"]

1. Record & Cryptographically Verify an Agent Trajectory (FlightRecorder)

from agentflight import FlightRecorder, CassetteReplayer, verify_cassette

recorder = FlightRecorder(agent_id="sre-incident-agent", metadata={"env": "prod"})

recorder.record(
    kind="llm_turn",
    actor="planner_model",
    input_payload={"goal": "Investigate payment-api p99 latency spike"},
    output_payload={"next_tool": "prometheus.query_metrics"},
    latency_ms=410.0,
    prompt_tokens=580,
    completion_tokens=90,
    cost_usd=0.0022,
)
recorder.record(
    kind="mcp_rpc",
    actor="prometheus.query_metrics",
    input_payload={"service": "payment-api", "window": "15m"},
    output_payload={"p99_latency_ms": 1840.0, "anomaly": True},
    latency_ms=85.0,
)

# Persist as tamper-evident .flight.json or .flight.jsonl
recorder.save("cassettes/incident_8841.flight.json")
assert verify_cassette(recorder) is True

2. Replay Deterministically in <0.02 ms with $0.00 Spend (CassetteReplayer)

replayer = CassetteReplayer("cassettes/incident_8841.flight.json", strict_order=True)

# Stub any MCP or tool callable directly in pytest:
query_metrics_stub = replayer.stub_tool("prometheus.query_metrics", kind="mcp_rpc")

# Replaying matching steps returns recorded payloads in ~2.8 microseconds:
replayer.replay_step("llm_turn", "planner_model", {"goal": "Investigate payment-api p99 latency spike"})
result = query_metrics_stub(window="15m", service="payment-api")  # Key-order invariant!
assert result["anomaly"] is True

If a code change causes the agent to call the wrong tool or pass altered parameters, CassetteReplayer immediately raises TrajectoryDivergenceError with missing_keys, extra_keys, and changed_values.

3. Lint MCP Tool Schemas Across 10 Ergonomics Rules (ToolSmellLinter)

from agentflight import ToolSmellLinter, SHOWCASE_FLAWED_MCP_TOOLS

linter = ToolSmellLinter()
report = linter.lint(SHOWCASE_FLAWED_MCP_TOOLS)

print(f"Ergonomics Score: {report.ergonomics_score}/100 (Grade {report.grade})")
for finding in report.findings:
    print(f"[{finding.rule_id}] {finding.tool_name}: {finding.message}")
    print(f"  Fix -> {finding.remediation}")

4. Guard Live Agents Against Loops, Ping-Pong Cycles & Runaway Spend (TrajectoryCircuitBreaker)

from agentflight import TrajectoryCircuitBreaker, CircuitBreakerTripped

breaker = TrajectoryCircuitBreaker(
    max_identical_repeats=3,
    max_ping_pong_cycles=2,
    max_consecutive_errors=3,
    max_total_tokens=25_000,
    max_cost_usd=0.50,
)

try:
    breaker.observe("search_logs", {"query": "timeout"})
    breaker.observe("refine_query", {"query": "timeout"})
    breaker.observe("search_logs", {"query": "timeout"})
    breaker.observe("refine_query", {"query": "timeout"})  # Trips on A -> B -> A -> B!
except CircuitBreakerTripped as exc:
    print(exc.reason, exc.step_index, exc.evidence)
    # -> ping_pong_oscillation 3 {'period': 2, 'cycles': 2, 'pattern_actors': ['search_logs', 'refine_query']}

5. Compare Candidate vs. Golden Trajectories (compare_cassettes & TrajectoryEvaluator)

from agentflight import compare_cassettes, diff_trajectories

# v1.1.0 Trajectory Differential Analyzer (Levenshtein/LCS sequence diff + loop/error/cost regression check)
report = compare_cassettes("cassettes/baseline.flight.jsonl", "cassettes/candidate.flight.jsonl")
print(report.regression_detected, report.similarity_score, report.added_tools, report.cost_delta_pct)
print(report.summary_markdown)

# Golden Evaluation Metrics (Precision, Recall, F1, Step Efficiency)
diff = diff_trajectories(golden_recorder, candidate_recorder)
print(diff.tool_precision, diff.tool_recall, diff.step_efficiency_ratio, diff.goal_completion_score)

6. Export W3C TraceContext / OpenTelemetry Spans for stackpulse (export_cassette_to_otel)

from agentflight import export_cassette_to_otel

spans = export_cassette_to_otel(
    "cassettes/incident_8841.flight.json",
    service_name="sre-incident-agent",
    output_path="spans.json",
)
# Directly ingestible by `codesphere-u/stackpulse` for critical-path DAG analysis and SLO burn-rate monitoring!

MCP Tool-Smell Rule Catalog (TS001–TS010)

Measured Benchmark Results

Reproducible via python -m agentflight.cli benchmark (measured on Python 3.12):

CLI Reference

# 1. Lint showcase flawed vs. hardened MCP tool suites (or pass --schema-file path/to/tools.json)
agentflight lint

# 2. Automated Schema Repair: harden smelly tools to Glama TDQS Gold Standard
agentflight fix tools.json --output hardened_tools.json

# 3. CI/CD Pre-Flight Gate: lint tool smells, verify cassette integrity, enforce budget SLAs
agentflight ci --schema-file tools.json --cassette runs/test.cassette.json --fail-on-smell

# 4. Forensic Post-Mortem Investigator: generate AAIB/NTSB incident reports from flight cassettes
agentflight post-mortem crash_incident.cassette.json --output report.md

# 5. Run the end-to-end recording, <0.02ms replay, circuit breaker & diff demo
agentflight demo

# 6. Run the reproducible subsystem benchmark suite
agentflight benchmark

# 7. Compare baseline vs. candidate cassettes for behavioral regressions
agentflight diff baseline.flight.json candidate.flight.json --json

# 8. Export cassette to W3C TraceContext / OpenTelemetry JSON spans for StackPulse
agentflight export-otel incident_8841.flight.json --output spans.json

# 9. Start the JSON-RPC 2.0 MCP Server over stdio
agentflight serve-mcp

Deterministic Testing with @flight_test (VCR for AI Agents)

Eliminate non-determinism, API cost, and network latency from your CI test suite:

import pytest
from agentflight.testing import flight_test

@flight_test("tests/fixtures/checkout.cassette.json")
def test_checkout_agent(flight):
    # During first run: executes live and records cryptographic cassette.
    # During subsequent runs (e.g. CI): replays in <3 μs with $0.00 network cost!
    agent = OrderAgent()
    result = agent.execute("Order item sku_881 with coupon SAVE10")
    
    assert result.status == "confirmed"
    assert flight.step_count == 3
    assert flight.total_cost_usd <= 0.05

MCP Tools Reference & Usage Guide

AgentFlight provides 4 specialized MCP tools for AI coding agents and LLM tool designers:

Tool Usage Examples

1. agentflight_lint_tools — Auditing MCP Tool Ergonomics Before Deployment

AI agent frameworks often suffer from confusing tool descriptions, ambiguous parameters, or overlapping tools. Call agentflight_lint_tools to validate your schemas before registering them:

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "agentflight_lint_tools",
    "arguments": {
      "tools": [
        {
          "name": "search_database",
          "description": "Searches customer database by email address or user ID. Returns matched records.",
          "inputSchema": {
            "type": "object",
            "properties": {
              "query": { "type": "string", "description": "Email or user ID string." }
            },
            "required": ["query"]
          }
        }
      ]
    }
  }
}

2. agentflight_diff_cassettes — Detecting Agent Behavioral Regressions in CI

When updating prompts or upgrading from GPT-4o to Claude 3.5 Sonnet, verify that the new agent's tool execution trajectory does not deviate or loop:

{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/call",
  "params": {
    "name": "agentflight_diff_cassettes",
    "arguments": {
      "baseline_path": "fixtures/golden_deploy_workflow.flight.json",
      "candidate_path": "test_runs/candidate_deploy.flight.json",
      "similarity_threshold": 0.85,
      "max_cost_increase_pct": 20.0
    }
  }
}

Exposed MCP Resources (resources/list, resources/read)

Exposed MCP Prompts (prompts/list, prompts/get)

Architectural Decision Records (ADRs)

ADR-001: Canonical JSON SHA-256 Chaining for Trajectory Cassettes

• Context: Golden test fixtures for multi-step agents frequently suffer from dictionary key-order drift across Python/JSON runtimes and silent fixture edits.
• Decision: Every TrajectorySpan computes input_hash over json.dumps(..., sort_keys=True, separators=(",", ":")) and chains step_hash = SHA256(step_index || kind || actor || input_payload || output_payload || status || prev_hash).
• Consequence: Reordered keyword arguments match deterministically during replay while any mutation to recorded inputs, outputs, or step ordering triggers CassetteTamperError.

ADR-002: Zero-Dependency Static Ergonomics Linting (TS001–TS010)

• Context: Calling an LLM judge to critique MCP tool definitions is slow, non-deterministic in CI, and requires API credentials.
• Decision: Implement a fast, deterministic 10-rule static analyzer combining JSON Schema AST traversal with tokenized bag-of-words cosine similarity (TS008).
• Consequence: 200 MCP tool definitions are linted in ~119 ms (1,676 tools/sec) with zero external dependencies.

Hosted endpoint (Vercel)

A free, public, stateless deployment of AgentFlight runs over MCP Streamable HTTP:

curl -s -X POST https://agentflight-tau.vercel.app/mcp -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

Client Configuration

1. Claude Code CLI (Recommended)

# Add to current project:
claude mcp add --transport http agentflight https://agentflight-tau.vercel.app/mcp

# Or add globally across all projects:
claude mcp add --scope user --transport http agentflight https://agentflight-tau.vercel.app/mcp

2. Claude Desktop / Cursor / Windsurf (mcp.json)

{
  "mcpServers": {
    "agentflight": {
      "type": "http",
      "url": "https://agentflight-tau.vercel.app/mcp"
    }
  }
}

3. Smithery Registry (1-Click Install)

npx -y @smithery/cli install @codesphere-u/agentflight --client claude

4. Local Stdio via uvx (Full Local File System Access)

{
  "mcpServers": {
    "agentflight": {
      "command": "uvx",
      "args": [
        "--from",
        "agentflight-mcp",
        "agentflight",
        "serve-mcp"
      ]
    }
  }
}

Hosted limitations: Only agentflight_lint_tools is exposed. The cassette tools (agentflight_diff_cassettes, agentflight_export_otel, agentflight_verify_cassette) read/write local files and are disabled on the hosted endpoint — use the local stdio install.

The hosted gateway is api/index.py; it enforces a 1 MB body limit, strict JSON-RPC envelopes and the tool restrictions above. Deploy your own copy with npx vercel deploy --prod (entrypoint is configured in pyproject.toml under [tool.vercel]).

Official OpenAny Provider Integration

AgentFlight features an official commercial provider integration for OpenAny, giving developers access to 128+ AI models and 3,200+ external APIs with pre-execution cost circuit breaking, black-box flight recording, and sub-microsecond deterministic replay.

🤝 Partner Attribution & Transparent Disclosure:
AgentFlight integrates natively with OpenAny. Outbound requests to OpenAny carry the verified partner header X-OpenAny-Partner-Id: agentflight.
Disclosure: AgentFlight may earn a commission from OpenAny usage through our partnership, at no additional cost to you where confirmed by OpenAny.
AgentFlight remains 100% provider-agnostic, open-source, and free to use with any model provider.

Quick Start with OpenAny

from agentflight import OpenAnyProvider, OpenAnyConfig, TrajectoryCircuitBreaker, FlightRecorder

# Initialize provider with spending limits
provider = OpenAnyProvider(
    config=OpenAnyConfig(
        api_key="your_openany_api_key",
        default_model="MiniMax-M2.5",
        max_budget_usd=0.05,
    )
)

guard = TrajectoryCircuitBreaker(max_cost_usd=0.02)
recorder = FlightRecorder(agent_id="openany-agent")

# Pre-execution budget shield automatically trips BEFORE any paid request if cost exceeds limits
resp = provider.chat_completion(
    messages=[{"role": "user", "content": "Summarize distributed consensus"}],
    guard=guard,
    recorder=recorder,
)

# Save tamper-evident cassette for $0.00 CI testing
recorder.save("session.flight.json")

CLI Commands for OpenAny

# List all 128+ models and real-time per-token prompt/completion pricing
agentflight openany models

# Run a model prompt with budget guardrail and flight recording
agentflight openany chat --model MiniMax-M2.5 --prompt "Explain Paxos" --budget 0.01 --record paxos.flight.json

# Audit OpenAny remote MCP tools against 10 ergonomics rules (TS001-TS010)
agentflight openany lint-tools

# Deterministically replay recorded trajectory in 0.08 μs with zero API spend
agentflight openany replay --cassette paxos.flight.json

For complete technical specifications, see docs/OPENANY_INTEGRATION.md, docs/COMMERCIAL_PARTNERSHIP.md, and docs/LIVE_ATTRIBUTION_RECORD.md.

Authorship, Ownership & License

AgentFlight is independently created and owned by Akandwanaho Ivan Alvin.
It is not owned or contracted by Codesphere Innovations (U) Limited merely because it is hosted under the codesphere-u GitHub organization.

• Creator & Owner: Akandwanaho Ivan Alvin (GitHub @codesphere-u)
• Project Status: Independently owned open-source AI infrastructure
• Commercial Partner: OpenAny
• License: MIT License — Copyright (c) 2026 Akandwanaho Ivan Alvin (codesphere-u).

If AgentFlight saves your agents from runaway token loops or speeds up your CI suite, please ⭐ Star this repository and share it with fellow AI builders!