Skip to main content

AgentTrace

AgentTrace

AgentTrace is a lightweight and hackable tracing/evaluation framework for AI agents and language models. It provides local monitoring and debugging capabilities, making it easier to build reliable and performant AI systems.

Installation

You can install AgentTrace directly from PyPI:

pip install agenttrace

Or install from source:

git clone https://github.com/tensorstax/agenttrace.git
cd agenttrace
pip install -e .

Quick Start

Basic Tracing

from agenttrace import TraceManager, TracerEval

tracer = TraceManager(db_path="traces.db")

@tracer.trace(tags=["test", "synchronous"], session_id="simple-function-test")
def test_function(test_input: str):
    return test_input

test_function("Hello, world!")

Tracing Async OpenAI API Calls with Tools

from openai import AsyncOpenAI
import json

get_capital_tool = {
    "type": "function",
    "function": {
        "name": "get_capital",
        "description": "Returns the capital city of a specified country",
        "parameters": {
            "type": "object",
            "required": ["country"],
            "properties": {
                "country": {
                    "type": "string",
                    "description": "The name of the country for which to find the capital"
                }
            },
            "additionalProperties": False
        },
        "strict": True
    }
}

@tracer.trace(tags=["async", "openai", "tool-calling"], session_id="simple-openai-tool-calling-test")
async def create_async_chat_completion(messages, model="gpt-4o", temperature=1, max_tokens=2048, tools=None):
    client = AsyncOpenAI()
    response = await client.chat.completions.create(
        model=model,
        messages=messages,
        response_format={"type": "text"},
        tools=tools,
        temperature=temperature,
        max_completion_tokens=max_tokens,
        top_p=1,
        frequency_penalty=0,
        presence_penalty=0,
        store=False
    )
    if response.choices[0].message.tool_calls:
        return json.loads(response.choices[0].message.tool_calls[0].function.arguments)
    return response.choices[0].message.content

response = asyncio.run(create_async_chat_completion(
    [{"role": "user", "content": "What is the capital of France?"}],
    tools=[get_capital_tool]
))
print(response)
# You can now view the traces in the web interface with: agenttrace start

Using the Evaluation Framework

from agenttrace import TracerEval
import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI()

async def get_capital(input_message: str):
    response = await client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": input_message}
        ]
    )
    return response.choices[0].message.content

def capital_checker(output):
    return {"score": 1.0 if "paris" in output.lower() else 0.0}

capital_checker.name = "capital_checker"

async def main():
    evaluator = TracerEval(
        name="france_capital_test",
        data=lambda: [{"input": "What is the capital of France?"}],
        task=get_capital,
        scores=[capital_checker]
    )
    
    results = await evaluator.run()
    print(f"Accuracy: {results['eval_results'][0]['scores']['capital_checker']['score']}")

if __name__ == "__main__":
    asyncio.run(main())

Web Interface

agenttrace includes a web-based interface for visualizing traces and evaluation results.

Starting the Web Interface

# Or navigate to the agenttrace/frontend directory
cd agenttrace/frontend

# Install dependencies if this is your first time
npm run install:all

# Start both the backend API and frontend interface
npm run start

This will start:

  • The backend API server on port 3033
  • The frontend web interface on port 5173

Open your browser and go to http://localhost:5173 to access the interface.

Customizing Trace Storage

By default, AgentTrace stores traces in a SQLite database at traces.db in the current directory. You can customize this:

from agenttrace import TraceManager

# Use a custom database path
tm = TraceManager(db_path="/path/to/custom/traces.db")

Adding Custom Tags

Tags help you categorize and filter traces:

# Add tags to traces
tm.add_trace("START", "custom_operation", tags=["important", "production", "v2"])

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Metadata

Release files for agenttrace 0.1.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for agenttrace 0.1.2
File Size Uploaded Provenance Actions
agenttrace-0.1.2.tar.gz 17.4 kB Without provenance Details for agenttrace-0.1.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for agenttrace 0.1.2
File Interpreter ABI Platform Actions
agenttrace-0.1.2-py3-none-any.whl Python 3 none any Details for agenttrace-0.1.2-py3-none-any.whl

Total release size: 32.7 kB

Release files / agenttrace-0.1.2.tar.gz

Download URL agenttrace-0.1.2.tar.gz
Size 17.4 kB
Tags Source
SHA-256 checksum
How to use checksums
cf06a5ad4e5dd5f174a64025b4e3d5b1fa13ee0afb58cd787902d1943919d083
BLAKE2b-256 checksum
How to use checksums
0817cca63e3a9a9add0e1dd82ca75ed916de0dc2499900a60731c6db81bde2d3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.6.11

Release files / agenttrace-0.1.2-py3-none-any.whl

Download URL agenttrace-0.1.2-py3-none-any.whl
Size 15.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
da68935d3b58756498073733a85f647265255b6cb6d46d2ff81c5b28ef8af568
BLAKE2b-256 checksum
How to use checksums
dbf5b7177f6067ce49bf02943c8bbaebf82537930ab156f80a217ec01722f454
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.6.11

Security for agenttrace 0.1.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page