toolproxy 0.4.0

Universal tool-calling wrapper for non-tool-native LLMs — emulates function calling via structured JSON planning

toolproxy

Universal Tool-Calling Wrapper for Non-Tool-Native LLMs

A provider-agnostic Python library that adds reliable tool/function calling to any LLM — even models that have no native tool-calling API.

---

Problem

Many LLM providers (OpenRouter, Ollama, local LLMs) expose models that don't support function calling. This library solves that by:

• Detecting whether the model supports native tool calling.
• Using native tool calls when available (OpenAI format).
• Falling back to a structured JSON planning protocol when not.

The developer always uses the same API regardless of the underlying model.

---

Installation

pip install toolproxy

Or from source:

pip install -e ".[dev]"

---

Quick Start

from toolproxy import UniversalAgent, tool

@tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"Sunny, 25°C in {city}"

agent = UniversalAgent(
    model="openrouter/mistralai/mistral-7b-instruct",
    tools=[get_weather],
)

result = agent.run("What is the weather in Chennai today?")
print(result.content)

The same code works whether the model supports native tools or not.

---

How It Works

Developer
    │
    ▼
UniversalAgent.run(prompt)
    │
    ├─ Planner (auto-detects native vs emulated mode)
    │       │
    │       ├── Native mode  → provider tool calls (OpenAI format)
    │       └── Emulated mode → structured JSON Action schema
    │
    ├─ Executor (validates args, runs tool, captures errors)
    │
    └─ LoopController (repeats until final answer or max_steps)

---

Model Prefixes

Prefix	Backend
openrouter/...	OpenRouter API
ollama/...	Local Ollama server
mock/...	MockClient (for testing, no API key needed)
(no prefix)	OpenAI / any OpenAI-compatible endpoint

---

Advanced Options

from toolproxy import UniversalAgent, tool
from toolproxy.config import ExecutionPolicy

agent = UniversalAgent(
    model="openrouter/your-model",
    tools=[get_weather],
    mode="auto",               # "auto" | "native_only" | "emulated_only"
    max_steps=10,
    execution_policy=ExecutionPolicy(
        mode="allow_only",
        allowed_tools=["get_weather"],
    ),
)

result = agent.run("...", return_trace=True)
print(result.content)
for call in result.trace.tool_calls:
    print(call.tool_name, call.arguments)

Callbacks (streaming-style)

result = agent.run(
    "...",
    on_tool_call=lambda step, tc: print(f"Calling: {tc.tool_name}"),
    on_tool_result=lambda step, tr: print(f"Result: {tr.output}"),
    on_model_output=lambda step, text: print(f"Model: {text}"),
)

---

Emulated Mode Protocol

When the model does not support native tools, the agent injects a system prompt instructing the model to output one of two JSON formats:

// Tool call
{"type": "tool_call", "tool": {"tool_name": "get_weather", "arguments": {"city": "Chennai"}}}

// Final answer
{"type": "final", "content": "The weather is sunny."}

Malformed responses are retried up to parse_retries times (default: 3) with an error explanation.

---

Project Structure

src/toolproxy/
  __init__.py      # Public API re-exports
  agent.py         # UniversalAgent class
  llm_client.py    # LLMClient + adapters
  tools.py         # @tool decorator + ToolRegistry
  schemas.py       # Pydantic schemas
  planner.py       # Planner logic
  executor.py      # Tool execution + policies
  loop.py          # Loop controller
  exceptions.py    # Custom exceptions
  config.py        # Configuration + capability map
examples/
  basic_chat.py
  openrouter_tools.py
  local_ollama.py
tests/
  test_agent_basic.py
  test_emulated_mode.py
  test_native_mode.py
  test_error_handling.py
  test_tool_registry.py

---

Publishing to PyPI

# 1. Install build tools
pip install build twine

# 2. Build wheel + sdist
python -m build

# 3. Check the distribution
twine check dist/*

# 4. Upload to PyPI (you will be prompted for credentials)
twine upload dist/*

# Or upload to TestPyPI first
twine upload --repository testpypi dist/*

---

Running Tests

pytest tests/ -v

---

Environment Variables

Variable	Description
OPENROUTER_API_KEY	API key for OpenRouter
OPENAI_API_KEY	API key for OpenAI
OLLAMA_BASE_URL	Ollama server URL (default: http://localhost:11434)
OLLAMA_MODEL	Ollama model name (default: llama3)