humanbound 2.11.0

humanbound

Open-source adversarial testing engine, SDK, and CLI for AI agents.
Attack your agent the way real users and attackers will: live endpoints, multi-turn conversations, tool abuse. Then turn every failure into a firewall rule.
Runs locally or against the Humanbound Platform. No login required to start.

Quick Start · Test-to-Guardrail Loop · SDK · Documentation · Contributing

📖 Full documentation lives at docs.humanbound.ai — this README covers the essentials; the docs have the depth.

Why Humanbound

Most testing tools test prompts. Humanbound tests agents: it drives multi-turn conversations against your real endpoint, probes tool use and scope boundaries, and scores the results against your security policy. When tests fail, hb guardrails converts the findings into deployable firewall rules — so the same run that finds a hole also patches it.

Quick Start

Install

pip install humanbound                       # CLI + SDK, core deps
pip install humanbound[engine]               # + OpenAI / Anthropic / Gemini / Ollama providers
pip install humanbound[firewall]             # + humanbound-firewall runtime
pip install humanbound[engine,firewall]      # everything

Docker

Run the CLI from the official image (docs):

docker run --rm -v "$PWD":/workspace \
  -e HB_PROVIDER=openai -e HB_API_KEY=$OPENAI_API_KEY -e HB_MODEL=gpt-4o-mini \
  ghcr.io/humanbound/humanbound:2 test --endpoint ./bot-config.json --wait

CLI usage

# Configure your LLM provider
export HB_PROVIDER=openai
export HB_API_KEY=sk-...

# Point Humanbound at your agent and run an adversarial test
hb test --endpoint ./bot-config.json --repo . --wait

# View results
hb posture                         # security score (0-100, A-F)
hb logs                            # full multi-turn conversation logs
hb report -o report.html           # HTML report

Full air-gap with Ollama — zero external API calls (requires the [engine] extra):

pip install humanbound[engine]
export HB_PROVIDER=ollama
export HB_MODEL=llama3.1:8b
hb test --endpoint ./bot-config.json --scope ./scope.yaml --wait

Describe your agent

bot-config.json tells the engine how to call your agent's API:

{
  "chat_completion": {
    "endpoint": "https://your-bot.com/chat",
    "headers": {"Authorization": "Bearer <token>"},
    "payload": {"message": "$PROMPT"}
  },
  "thread_init": {
    "endpoint": "https://your-bot.com/sessions",
    "headers": {"Authorization": "Bearer <token>"},
    "payload": {}
  }
}

• chat_completion (required) — the request the engine sends for every conversation turn: your agent's chat endpoint.
• thread_init (optional) — called once before each conversation to create a session/thread. Stateless agents simply omit it (same for thread_auth, an optional separate auth step).

The headers and payload above are examples — they must match your agent's actual API input schema. The engine sends them as-is after substituting the placeholders:

• $PROMPT — replaced with the user message of each attack turn. If no $PROMPT appears in the payload, the engine appends the message as an OpenAI-style messages array instead.
• $CONVERSATION — replaced with the prior turns of the conversation, for stateless agents that expect the full history in every request.

scope.yaml declares what the agent is — and is not — allowed to do. The restricted list is what turns generic jailbreak probes into targeted tool-abuse attacks:

business_scope: "Customer support for Acme Bank"
permitted:
  - Provide account balance and transaction info
  - Process routine transfers within limits
restricted:
  - Process transfers above 10,000 EUR   # tool-abuse boundary the engine attacks
  - Access internal system records
more_info: "HIGH: finance domain agent"

See Agent Configuration and Scope Discovery for the full specifications.

From test results to guardrails

Every adversarial run produces training data. Feed it straight back into your defenses:

hb test --endpoint ./bot-config.json --wait   # find the failures
hb guardrails -o rules.json                   # convert findings into guardrail rules
hb guardrails -f yaml -o agent.yaml           # the firewall's policy file
hb firewall train --model detectors/setfit_classifier.py   # train a Tier 2 classifier

Deploy the output with humanbound-firewall and your agent is protected against exactly the attacks it just failed. No other open-source tool closes this loop.

Python SDK

import json
import time

from humanbound import LocalRunner, TestConfig

config = TestConfig(
    endpoint=json.load(open("bot-config.json")),  # same config file the CLI uses
    scope_path="scope.yaml",
)

runner = LocalRunner()
experiment_id = runner.start(config)

while runner.get_status(experiment_id).status not in ("Finished", "Failed", "Terminated"):
    time.sleep(5)

posture = runner.get_posture(experiment_id)
print(f"Security posture: {posture.overall_score} ({posture.grade})")

for insight in runner.get_result(experiment_id).insights:
    print(insight["severity"], "—", insight["category"])

The CLI and SDK share the same implementation, so they cannot drift. Authoring custom orchestrators and wiring EngineCallbacks is covered in the docs.

Architecture

The CLI funnels every entry point — hb commands and the MCP server — through a single TestRunner abstraction. A PlatformTestRunner forwards to the Humanbound backend; a LocalTestRunner drives the in-process engine (scope discovery, orchestrators, bot, judge, presenter) against your own LLM provider, with results written under .humanbound/results/.

Stability contract

The full orchestrator authoring guide, Platform integration, and API reference live on docs.humanbound.ai.

Release highlights

• Clean name: humanbound is the PyPI install. The old humanbound-cli package has been yanked from PyPI; install humanbound directly.
• Public SDK namespace alongside the CLI — use the CLI or drive the engine from Python.
• Firewall integration: pip install humanbound[firewall] pulls humanbound-firewall alongside the CLI.

See the changelog for full release notes.

Contributing

Contributions welcome. See CONTRIBUTING.md for the dev loop, release process, and DCO sign-off requirement (see DCO.md).

• 🐛 Report a bug
• 💡 Request a feature
• 🔒 Report a security issue — not via public Issues
• 💬 Join Discord

Telemetry

The hb CLI sends anonymous usage data to help us improve it. Disable with hb telemetry disable, HB_TELEMETRY_DISABLED=1, or DO_NOT_TRACK=1. Turning telemetry off sends one final anonymous telemetry_disabled event (once per machine, ever) so we can count opt-outs. Full disclosure: PRIVACY.md.

License

Apache-2.0. Free to use in any context — commercial or open-source — with attribution. See TRADEMARK.md for the trademark policy. The code is open; the name is not.

The sibling project humanbound-firewall is also Apache-2.0 — same license, different product.