bastion-red

Adversarial assessment SDK for AI agents. Python equivalent of @pistonsolutions/bastion. Same CLI, same wrap() programmatic API, same scope authoring, same hosted-runner endpoints.

pipx install bastion-red          # or: uv tool install bastion-red
bastion login
bastion run --target https://your-agent.example.com/chat --plugin pii-direct

The same run appears in your dashboard with the full probe target transcript.

Full reference: https://bastion.pistonsolutions.ai/docs

Quickstart

pipx install bastion-red
bastion login                 # opens dashboard, hands API key back to terminal
bastion init                  # scaffold .bastion/ in the current repo
bastion run --scope SCOPE.md  # text assessment
# or:
bastion voice --target-ws wss://your-agent.example.com/voice \
              --rule "Do not disclose PII without verification"

Requires Python ≥ 3.10.

Architecture

The SDK is a thin client. Your machine never runs an LLM, opens a WebSocket to your target, or places a phone call — all of that happens on Bastion infra. The CLI reads your scope, POSTs a job descriptor with your bk_* token, polls until the run completes.

What this means: only your target's URL/DID and the scope leave your network. No LLM keys on your end, no telephony provider to integrate, no WebRTC stack to install.

Failsafe: if the primary probe engine fails on a given run, Bastion falls back to a curated payload catalog automatically. You get telemetry from every bastion run, not an empty report.

Modes

bastion run --target https://your-agent.example.com/chat --plugin pii-direct
bastion voice --target-ws wss://your-agent.example.com/voice --rule "..."
bastion voice --target-did +14155551234 --rule "..."

Telephony note: only test phone numbers you own or have explicit written authorization to test.

Programmatic API: wrap()

from bastion import wrap

def agent(msg: str) -> str:
    return llm.complete(msg)

traced = wrap(agent, mode="observe", on_event=lambda e: print("event", e["session_id"]))
traced("hello")

Events flow into the dashboard's Live Activity view. Flagged events auto-promote into scope-seeds so the next assessment can replay them as adversarial probes.

SCOPE.md

YAML frontmatter + markdown body. Frontmatter declares what to test; body describes the agent for grading context.

---
target: https://api.example.com/v1/chat/completions
model: gpt-4o-mini

headers:
  Authorization: Bearer ${OPENAI_API_KEY}

categories:
  - system_prompt_extraction
  - pii_leakage
  - jailbreak_resistance
---

# Customer-support agent

The agent answers billing questions for retail customers. Must verify identity
(email + last 4 of card) before discussing any account, must refuse refunds
above $100 without supervisor approval.

Env vars in headers interpolate via ${VAR}. Categories default to system_prompt_extraction if omitted.

CLI reference

Every command supports --help.

Authentication

Two ways to provision a bk_* key:

1. Interactive: bastion login (opens browser, hands key back, persists ~/.bastion/credentials.json mode 0600)
2. CI: export BASTION_API_KEY=bk_live_...

Resolution order: env -> file. Keys revocable from the dashboard.

CI/CD

name: Bastion Assessment
on: { pull_request: {}, push: { branches: [main] } }

jobs:
  bastion:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.12' }
      - run: pipx install bastion-red
      - run: bastion run --scope SCOPE.md --no-tui
        env: { BASTION_API_KEY: ${{ secrets.BASTION_API_KEY }} }

Privacy

• Only the scope you write leaves your machine.
• Verbatim target responses are stored as evidence — if your agent returns real PII, that PII is visible in the dashboard transcript.
• API keys never leave the dashboard in plaintext (only a SHA-256 hash).
• Runs are scoped to the org that minted the API key.

MIT © Piston Solutions