conduct-litellm-guard

Conduct Guard as a LiteLLM guardrail. Every LLM call routed through LiteLLM runs through your active Guard policy before the request leaves your network — block, warn, audit, or trigger a HITL approval, with the same signed configuration and hash-chained audit log as the rest of Conduct.

Install

pip install conduct-litellm-guard

Wire it up

For optional user-delegated identity, see Federation setup. It uses a separate service credential and trusted custom-auth handoff; the service-only setup below is unchanged.

1. Mint an agent token in the Conduct console (Settings → Agent identities). Copy the cond_agt_… value.

2. Export it in the environment where your LiteLLM proxy runs:

   export CONDUCT_AGENT_TOKEN=cond_agt_...
   

3. Add the guardrail to your LiteLLM proxy's config.yaml. Two forms — both work today, they differ only in the guardrail: string:

   Explicit (works on any LiteLLM version):

   guardrails:
     - guardrail_name: conduct-guard
       litellm_params:
         guardrail: conduct_litellm_guard.ConductGuard
         mode: [pre_call, pre_mcp_call]
         api_url: https://gateway.conductai.ai
         agent_token: os.environ/CONDUCT_AGENT_TOKEN
         unreachable_fallback: fail_closed
   

   Native (LiteLLM with BerriAI/litellm#40785):

   guardrails:
     - guardrail_name: conduct-guard
       litellm_params:
         guardrail: conduct
         mode: [pre_call, pre_mcp_call]
         api_key: os.environ/CONDUCT_AGENT_TOKEN
         unreachable_fallback: fail_closed
   

   The native form uses LiteLLM's idiomatic api_base / api_key parameter names. You still install this package — pip install conduct-litellm-guard — because the native adapter is a thin wrapper that imports from it. Same pattern Aporia and Lakera use.

4. Start the proxy:

   litellm --config config.yaml
   

A full example lives in examples/config.yaml.

What the guardrail does

For every incoming request the guardrail calls Conduct's guard_check tool (JSON-RPC over the MCP endpoint) with a compact summary of the LiteLLM request — model, call type, message count, temperature, stream flag, plus the last user message so the audit trail carries context.

Guard returns one of five verdicts:

MCP tool calls (pre_mcp_call, 0.2.7+)

When LiteLLM's MCP gateway runs a tool, the guardrail also calls Guard's guard_check action gate with the real tool name and arguments, the same check the CLI hook uses. Rules on tool + arguments (block, approval) apply to MCP traffic. The prompt check still scans the arguments for credentials / PII. The most severe verdict wins. Drop pre_mcp_call from mode: to keep the guardrail LLM-only.

Configuration reference

Session tracking

Guard uses a session ID to correlate the pre-call check with any resume-verdict / HITL approval that follows. The adapter picks the first value it finds in this order:

1. litellm_metadata.trace_id
2. metadata.X-Conduct-Session-Id (explicit override — set this if you want deterministic control over the session boundary).
3. metadata.conduct_session_id.
4. A deterministic hash of the user field plus the first user message.

If none of the above are available and the caller sends no user or messages, session tracking is skipped (single-shot mode). Approvals still work — they just can't be resumed against a prior check.

Fail modes

• fail_closed (default) — blocks the call if Guard is unreachable or the policy evaluator errors. Matches Conduct's default posture across every other enforcement surface.
• fail_open — allows the call and logs a warning. Use during rollout or if Guard is a soft dependency for your setup.

License

conduct-litellm-guard is distributed under the Apache License 2.0, the same license as the rest of Conduct.

Links

• Conduct Guard — the policy engine.
• Conduct Router — Conduct's own LLM proxy.
• Source — this package lives under packages/conduct-litellm-guard/.
• Issues — file bugs and requests against the parent repo.