swarmtrace 0.3.0

pytest for AI agents — trace, debug and catch regressions in LLM swarms

Copyswarmtrace 🐝

pytest for AI agents — trace, debug, and catch regressions in LLM swarms.

Install

bashpip install swarmtrace

Quick Start

pythonfrom tracely import observe

@observe def my_agent(question): # your LLM call here return "answer"

my_agent("What is machine learning?")

Then view your traces:

bashswarmtrace

Multi-Agent Swarm Tracing

pythonfrom tracely import observe

@observe def researcher(q): return llm.chat(f"Research: {q}")

@observe def summarizer(text): return llm.chat(f"Summarize: {text}")

@observe def orchestrator(q): research = researcher(q) return summarizer(research)

orchestrator("What is AGI?")

Output:

[swarmtrace] ▶ orchestrator started (id=2b914f91) [swarmtrace] ▶ researcher started (id=ffbf1215) [swarmtrace] ✓ done: researcher | 3.4s | 7in/330out | $0.0013 [swarmtrace] ▶ summarizer started (id=4fc29468) [swarmtrace] ✓ done: summarizer | 0.8s | 338in/78out | $0.0005 [swarmtrace] ✓ done: orchestrator | 4.2s | 7in/78out | $0.0003

Async Support

pythonimport asyncio from tracely import observe

@observe async def async_researcher(q): return await llm.achat(q)

@observe async def async_orchestrator(q): results = await asyncio.gather( async_researcher(q), async_researcher(q + " deep dive") ) return " | ".join(results)

asyncio.run(async_orchestrator("What is quantum computing?"))

CLI Commands

bashswarmtrace # view traces (default: last 100) swarmtrace --limit 50 # view last 50 traces swarmtrace-replay # replay any trace by ID swarmtrace-export --format json # export to JSON swarmtrace-export --format csv # export to CSV

Regression Detection

bashpip install swarmtrace[regression]

pythonfrom tracely.regression import compare

def my_agent(input_text, prompt): return llm.chat(f"{prompt}\n\n{input_text}")

compare( my_agent, inputs=["What is ML?", "How does Python work?", "What is an API?"], version_a_prompt="You are a helpful assistant.", version_b_prompt="Reply only in emojis.", threshold=0.6, # optional, default 0.6 )

Output:

INPUT V1 LATENCY V2 LATENCY SIMILARITY REGRESSION? What is ML? 3.7s 1.5s 0.1 🔴 YES How does Python work? 3.0s 1.1s 0.15 🔴 YES What is an API? 3.1s 1.0s 0.15 🔴 YES

Result: 3/3 regressions detected ⚠️ WARNING: Your new prompt may have regressed!

Remote Ingest (SaaS)

pythonfrom tracely import init, observe

init( api_key="your-swarmtrace-api-key", endpoint="https://your-swarmtrace-backend.com", )

@observe def my_agent(q): ...

Or via environment variables:

bashexport SWARMTRACE_API_KEY=your-key export SWARMTRACE_ENDPOINT=https://your-backend.com

Features

FeatureswarmtraceLangSmithOpen Source✅❌Works offline✅❌Any LLM✅❌ LangChain onlyMulti-agent tree✅✅Async support✅✅Regression detection✅❌One decorator setup✅❌Cost per agent✅✅Self-hosted✅❌Pricea DigitalOcean AMD Developer Cloud.

MetricValueHardwareAMD MI300X 192GBSFree$20/month

AMD MI300X Benchmarks

Tested on AMD Instinct MI300X GPU viwarms5 orchestratorsTotal agent calls20Avg orchestrator latency6.1sAvg researcher latency1.8sTrace overhead<1ms per call