benchspan 0.3.0

Prompt injection firewall for LLM agents. Scan tool outputs and user messages before they reach your model.

benchspan

Prompt injection firewall for LLM agents. One-line integration that scans tool outputs and user messages before they reach your model.

Install

pip install benchspan

Quick Start

from benchspan import BenchGuard

guard = BenchGuard(api_key="ag_live_...", agent="my-agent")

# Direct scan
result = guard.scan("some tool output", role="tool")
# result.injection  -> True/False
# result.verdict    -> "block", "warn", or "pass"

Framework Integrations

LangChain / CrewAI

from benchspan import BenchGuard
from langchain_anthropic import ChatAnthropic

guard = BenchGuard(api_key="ag_live_...")
llm = ChatAnthropic(model="claude-sonnet-4-6")
result = llm.invoke(messages, config={"callbacks": [guard]})

OpenAI Agents SDK

from benchspan import BenchGuard
from agents import Agent, Runner

guard = BenchGuard(api_key="ag_live_...")
agent = Agent(name="assistant", model="gpt-4o", hooks=guard.as_agent_hooks())
result = await Runner.run(agent, input="Hello")

Google ADK

from benchspan import BenchGuard
from google.adk.agents import LlmAgent

guard = BenchGuard(api_key="ag_live_...")
agent = LlmAgent(
    name="assistant",
    model="gemini-2.5-pro",
    before_model_callback=guard.as_adk_callback(),
)

Raw OpenAI / Anthropic SDK

from benchspan import BenchGuard
from openai import OpenAI

guard = BenchGuard(api_key="ag_live_...")
client = OpenAI()

@guard.wrap
def call_llm(messages):
    return client.chat.completions.create(model="gpt-4o", messages=messages)

result = call_llm(messages)

Modes

• "block" (default) -- Waits for scan result. Raises InjectionDetectedError if injection detected.
• "warn" -- Fires scan in the background with zero added latency. Logs a warning if injection detected but never blocks.

# Production -- block injections
guard = BenchGuard(api_key="ag_live_...", mode="block")

# Evaluation -- monitor without blocking
guard = BenchGuard(api_key="ag_live_...", mode="warn")

How It Works

1. Framework hook fires before each LLM call
2. SDK scans user and tool messages via the BenchSpan API
3. system and assistant messages are skipped (trusted)
4. Already-scanned messages are deduplicated across multi-turn conversations
5. In block mode, InjectionDetectedError is raised if injection detected
6. In warn mode, the scan runs async -- zero latency added to your agent

Links

• Dashboard
• Docs