FreeAgent SDK

A clean local agent SDK for Ollama, vLLM, and OpenAI-compatible servers.

Streaming. Multi-turn out of the box. Markdown skills and memory. Built-in telemetry. Single dependency.

pip install freeagent-sdk

Links: Documentation · Tutorial · Changelog · Contributing · Examples · Evaluation data

Why FreeAgent

• Local-first: works with Ollama and vLLM — your data never leaves your machine
• Streaming everywhere: token-level streaming with semantic events
• Multi-turn that just works: conversation state managed automatically with pluggable strategies
• Markdown is first-class: skills and memory are human-readable .md files with frontmatter
• Zero-config: auto-detects model size and tunes defaults — works on 2B and 70B alike
• Inspectable: agent.trace() shows exactly what happened
• Fast: actually 2% faster than raw Ollama API (HTTP connection reuse)
• Honest: real benchmark data in this README, not marketing

Quick Start

CLI

# One-shot query with streaming
freeagent ask qwen3:8b "What's the capital of France?"

# Interactive chat
freeagent chat qwen3:8b

# List available models
freeagent models

Python

from freeagent import Agent

agent = Agent(model="qwen3:8b")
print(agent.run("What is Python?"))

Streaming

Real token-by-token streaming, even for tool-using agents:

from freeagent import Agent
from freeagent.events import TokenEvent, ToolCallEvent, ToolResultEvent

agent = Agent(model="qwen3:8b", tools=[weather])

for event in agent.run_stream("What's the weather in Tokyo?"):
    if isinstance(event, TokenEvent):
        print(event.text, end="", flush=True)
    elif isinstance(event, ToolCallEvent):
        print(f"\n[Calling {event.name}...]")
    elif isinstance(event, ToolResultEvent):
        print(f"[{event.name} -> {'ok' if event.success else 'fail'} ({event.duration_ms:.0f}ms)]")

Async version: async for event in agent.arun_stream("query"):

Event types: RunStartEvent, TokenEvent, ToolCallEvent, ToolResultEvent, ValidationErrorEvent, RetryEvent, IterationEvent, RunCompleteEvent.

Custom Tools

from freeagent import Agent, tool

@tool
def weather(city: str) -> dict:
    """Get current weather for a city."""
    return {"city": city, "temp": 72, "condition": "sunny"}

agent = Agent(model="qwen3:8b", tools=[weather])
print(agent.run("What's the weather in Portland?"))

Multi-Turn Conversations

agent = Agent(model="qwen3:8b", tools=[weather])
agent.run("What's the weather in Tokyo?")
agent.run("Convert that to Celsius")  # remembers Tokyo was 85°F

Strategies

from freeagent import Agent, SlidingWindow, TokenWindow

# Default: SlidingWindow(max_turns=20)
agent = Agent(model="qwen3:8b")

# Token-based budget (better for small context models)
agent = Agent(model="qwen3:4b", conversation=TokenWindow(max_tokens=3000))

# Stateless mode (each run independent)
agent = Agent(model="qwen3:8b", conversation=None)

Session Persistence

agent = Agent(model="qwen3:8b", session="my-chat")
agent.run("Hello!")
# Later, in a new process:
agent = Agent(model="qwen3:8b", session="my-chat")  # restores conversation

Inspecting Runs

Every run is fully traced. See exactly what happened:

agent.run("What's 347 * 29?")

# One-line summary
print(agent.last_run.summary())
# Run 1: qwen3:8b (native) 2300ms, 2 iters, 1 tools

# Full timeline
print(agent.trace())
# +     0ms  model_call_start     iter=0
# +   800ms  tool_call            calc(expression='347*29')
# +   802ms  tool_result          calc -> ok (2ms)
# +   803ms  model_call_start     iter=1

# Markdown report
print(agent.last_run.to_markdown())

Model-Aware Defaults

FreeAgent auto-detects model capabilities from Ollama and tunes itself:

# Auto-tuned: detects 2B model, strips skills and memory tool
agent = Agent(model="gemma4:e2b")

# Auto-tuned: detects 8B model, keeps full defaults
agent = Agent(model="qwen3:8b")

# Override auto-tuning
agent = Agent(model="gemma4:e2b", bundled_skills=True, memory_tool=True)

# Disable auto-tuning entirely
agent = Agent(model="qwen3:8b", auto_tune=False)

Access detected info: agent.model_info.parameter_count, agent.model_info.context_length, agent.model_info.capabilities.

Skills (Markdown Prompt Extensions)

---
name: nba-analyst
description: Basketball statistics expert
tools: [search, calculator]
---

You are an NBA analyst. Always cite your sources.
When comparing players, use per-game averages.

agent = Agent(model="qwen3:8b", tools=[search, calculator], skills=["./my-skills"])

Bundled skills load automatically. User skills extend them — duplicate names override.

Memory (Markdown-Backed)

Every agent has built-in memory stored as human-readable .md files:

.freeagent/memory/
├── MEMORY.md          # Index
├── user.md            # auto_load: true → in system prompt
├── facts.md           # Accumulated facts
└── 2026-04-05.md      # Daily log

The agent gets a memory tool with actions: read, write, append, search, list. Only the index and auto_load files go into the system prompt — everything else is on demand.

Multi-Provider Support

from freeagent import Agent, VLLMProvider, OpenAICompatProvider

# vLLM
provider = VLLMProvider(model="qwen3-8b")
agent = Agent(model="qwen3-8b", provider=provider, tools=[my_tool])

# Any OpenAI-compatible server
provider = OpenAICompatProvider(model="llama3.1:8b", base_url="http://localhost:1234")
agent = Agent(model="llama3.1:8b", provider=provider, tools=[my_tool])

Telemetry

Built-in, always on:

agent.run("What's the weather?")
print(agent.metrics)               # quick summary
print(agent.metrics.tool_stats())  # per-tool breakdown
agent.metrics.to_json("m.json")   # export

Optional OpenTelemetry: pip install freeagent-sdk[otel]

MCP Support

from freeagent.mcp import connect

async with connect("npx -y @modelcontextprotocol/server-filesystem /tmp") as tools:
    agent = Agent(model="qwen3:8b", tools=tools)
    result = await agent.arun("List files in /tmp")

Install with: pip install freeagent-sdk[mcp]

Real Performance

Tested against the raw Ollama API with the same eval suite (100+ cases, 4 models). Full data in evaluation/.

Multi-Turn Conversations (6 conversations, 15 turns)

Tool Calling Accuracy (8 cases)

Streaming Latency (median of 3 runs)

TTFT ≈ total for chat (generation is fast once started). Tool TTFT includes tool execution round-trip.

Auto-Tune (v0.3.1)

Auto-tune detects gemma4:e2b as a small model and strips bundled skills + memory tool. This improves accuracy from 55% → 91%.

Honest Caveats

• Guardrails rarely fire: 0/40 real rescues in adversarial testing. Modern models handle fuzzy names and type coercion natively.
• Multi-turn gap to raw Ollama is noise: 87% vs 93% — re-running failures produces passes. Non-deterministic.
• Skills help qwen3:4b but hurt gemma4:e2b — fixed by auto-tune, which strips them for small models.
• Streaming TTFT ≈ total time on small models: generation is fast, model thinking dominates latency.

Full analysis: evaluation/THESIS_ANALYSIS.md

Tested Models

Requirements

• Python 3.11+
• Ollama running locally (ollama serve)
• A model pulled (ollama pull qwen3:8b)

Documentation

• Tutorial — 5-minute walkthrough from install to working agent
• Website — landing page and feature overview
• Examples — runnable scripts covering tools, memory, hooks, MCP
• Evaluation data — benchmark results and thesis analysis
• Changelog — release history
• Contributing — how to run tests, add skills, submit PRs

License

MIT