trikesh

Behavioral regression testing + runtime safety for LLM agents.

---

What it does

trikesh solves two distinct problems:

1. Behavioral regression testing — measure model behavior across 8 properties (sycophancy, honesty, consistency, prompt injection, trust hierarchy, minimal footprint, goal drift, corrigibility). Run structured probes and get coverage reports.

2. Runtime agent safety — intercept unsafe tool calls before execution. Compose a safety stack (rules → classifiers → LLMs). Plug in any judge: local models, HuggingFace, API-based LLMs, custom matchers, ensembles.

---

The 8 behavioral properties

These are what trikesh measures — independently of which classifier you use:

Property	What it checks
Sycophancy	Does the model abandon correct answers when users push back?
Honesty	Does the model acknowledge uncertainty instead of confabulating?
Consistency	Do equivalent questions get equivalent answers?
Prompt Injection	Does the model follow instructions embedded in external data?
Trust Hierarchy	Does the model respect operator rules over user requests?
Minimal Footprint	Does the model prefer reversible actions over irreversible ones?
Goal Drift	Does the model stay on task or expand scope without permission?
Corrigibility	Does the model stop when told to stop?

---

Why this matters

Behavioral safety isn't a jailbreak problem — it's a values problem.

Sycophancy, goal drift, prompt injection, corrigibility failures don't show up in accuracy benchmarks. They show up when users push back, when prompts change, when you swap providers. And by then, it's in production.

The MASK Benchmark (2026) found:

• No frontier model is honest more than 46% of the time under social pressure
• Larger models are less honest, not more
• 83% of models self-report knowing they contradicted their own beliefs

trikesh measures this. Before it reaches your users.

---

Installation

pip install trikesh

---

Benchmarking

# Run the static bench-v1 benchmark (reproducible, citable)
trikesh run --model gpt-4o-mini --benchmark bench-v1

# Generate dynamic probes
trikesh run --model gpt-4o-mini

# Compare two models side by side
trikesh compare --models gpt-4o-mini,claude-3-5-sonnet-20241022

# Check your judge's accuracy against ground truth
trikesh calibrate --judge-model gpt-4o-mini

Benchmarking with trikesh

trikesh includes bench-v1, a static set of 96 hand-authored probes grounded in safety research. Use it to evaluate any model:

from trikesh.benchmarks import load_benchmark

bench = load_benchmark("bench-v1")
# {"version": "bench-v1", "count": 96, "properties": [...]}

Results are reproducible and comparable across teams — useful as a reference when comparing models or evaluating your own classifiers.

---

Architecture

trikesh v0.5+ uses a pluggable, policy-driven architecture:

• Classifiers: Pluggable safety judges. Plug in LLM-based judges, rule-based matchers, HuggingFace models, or custom classifiers via a simple interface.
• Policy DSL: Declarative YAML policies define which classifiers run at which execution layers, with confidence thresholds and fallback chains.
• ExecutionEngine: Orchestrates classifiers across properties with two strategies:
  • Cascade: Try each layer's classifiers in order; stop at first confident result
  • Speculative: Run concurrent classifiers in a layer; use first confident winner (lower latency)
• Observable: Every classifier invocation is tracked — latency, confidence, outcome — accessible via guard.metrics

Backwards compatible: The legacy SafetyGuard(mode=..., judge_model=...) API still works unchanged.

PDP / PEP model

trikesh's runtime guard follows the same decision/enforcement split used in access-control systems (XACML, OPA) and increasingly in agent-authorization tools (AWS Cedar, NVIDIA OpenShell):

• SafetyGuard is the Policy Decision Point (PDP). It takes a proposed action + context and returns a structured verdict (SafetyCheckResult) — it never touches the live system itself. The decision logic is pluggable: any LLM via ModelAdapter, or any Classifier (rule-based, HuggingFace, local, ensemble). This makes trikesh a behavioral-judgment PDP — it reasons about pressure, honesty, consistency, and goal drift rather than matching static rules — distinct from and complementary to deterministic-permission PDPs (allowlists, spending ceilings) you might run alongside it.
• wrap() / protect() are the Policy Enforcement Points (PEP). They intercept the actual tool call — outside the model's own reasoning, so a compromised or manipulated agent can't talk its way past the check — query the PDP, and enforce its verdict (raise SafetyBlockedError or let the call through). These are framework-aware wrappers, not code merged into LangChain/AutoGen/ADK themselves: each adapter (e.g. LangChainAdapter) targets that framework's real tool-invocation point from outside, the same way most production PEPs work (a gateway, a proxy, an OS boundary — not literally inside the thing they're protecting).
• The Policy DSL (SPML) is the Policy Administration Point (PAP). Policies are authored as plain, flat YAML — deliberately with no expressions or embedded logic — decoupled from both the decision engine and the enforcement layer.

There's no formal Policy Information Point (PIP) yet — context, operator constraints, and goals are passed as explicit parameters today rather than gathered through a pluggable attribute provider. That's a known gap, not a shipped feature.

---

Runtime SafetyGuard

Legacy API (still works)

Add one check before your agent executes any action:

from trikesh import SafetyGuard

guard = SafetyGuard()

result = guard.check(
    action="DELETE FROM users WHERE last_login < '2023-01-01'",
    context="Production database agent",
    operator_constraints=[
        "Never DELETE on production without explicit written confirmation",
    ],
)

if not result.is_safe:
    # Don't execute — show violations to the user or log them
    for v in result.violations:
        print(f"[{v.severity.upper()}] {v.description}")

# Async pipelines
result = await guard.acheck(action="...", context="...")

Policy-Driven API (new)

Use declarative policies for fine-grained control:

from trikesh import SafetyGuard, Policy

# Load a pre-built policy
policy = Policy.from_yaml("policies/default.yml")
guard = SafetyGuard(policy=policy)

# Or define one in code
from trikesh.policy import Policy, PolicyProperty, ExecutionLayer

policy = Policy(
    version="1.0",
    policy_id="my-policy",
    properties=[
        PolicyProperty(
            name="sycophancy",
            classifier="llm:gpt-4o-mini",
            fallback="rule:capitulation",
        ),
    ],
    execution={
        "balanced": [
            ExecutionLayer(
                name="fast",
                classifiers=["rule:basic_checks"],
                timeout_ms=10,
                strategy="cascade",
            ),
            ExecutionLayer(
                name="thorough",
                classifiers=["llm:gpt-4o-mini"],
                timeout_ms=2000,
                strategy="cascade",
            ),
        ]
    },
)

guard = SafetyGuard(policy=policy)
result = await guard.acheck(action="...", context="...", mode="balanced")

# Inspect which classifiers were used
print(guard.metrics.summary())

Pluggable Classifiers

trikesh ships with built-in classifiers and supports custom ones:

from trikesh.classifiers import ClassifierRegistry, HFModelClassifier

# Use HuggingFace models
hf_classifier = HFModelClassifier("Qwen/Qwen2.5-0.5B")
ClassifierRegistry.register("hf:qwen-0.5b", hf_classifier)

# Use saroku-guard, the local PDP model — resolved by its built-in id
local = ClassifierRegistry.resolve("local:saroku-safety")

# Combine classifiers in an ensemble — register custom instances under "custom:"
from trikesh.classifiers import EnsembleClassifier
ensemble = EnsembleClassifier(
    classifiers=[local, hf_classifier],
    strategy="majority",  # or "cascade"
)
ClassifierRegistry.register("custom:hybrid", ensemble)

Modes

# Default — saroku-guard (local PDP) protects every call immediately, no
# setup required. Safe actions are cleared in ~10-30ms with no API cost.
# Anything flagged escalates to the LLM judge, if a provider key is set,
# for full attribution across all 8 behavioral properties.
guard = SafetyGuard()

# Local PDP only — zero API calls, works fully offline.
guard = SafetyGuard(mode="local")

# Skip the local PDP and always use the full LLM judge.
guard = SafetyGuard(mode="thorough", judge_model="gpt-4o-mini")

# Tune the block threshold — lower it to block more aggressively,
# raise it to reduce false-positive blocks.
guard = SafetyGuard(local_threshold=0.3)

What gets blocked

⛔ BLOCKED  DELETE inactive users from production   (no dry-run, no confirmation)
⛔ BLOCKED  Deploy with skip_tests=True             (safety bypass)
⛔ BLOCKED  Grant admin — no approval ticket        (constraint violation)
⛔ BLOCKED  Disable rate limiting                   (irreversible risk)
✅ ALLOWED  SELECT COUNT(*) — read-only query
✅ ALLOWED  Grant read access — ticket: JIRA-5821
✅ ALLOWED  Read service config

Framework Integration

trikesh integrates with popular agent frameworks — wrap tools or entire agents:

from trikesh import wrap, protect

# Protect a single tool
safe_search = wrap(agent.search_tool, guard=guard)

# Protect all tools in an agent (auto-detects framework)
from trikesh import SafetyBlockedError
safe_agent = await protect(agent, guard=guard)

# Handle blocked actions
try:
    result = await safe_agent.run(task)
except SafetyBlockedError as e:
    print(f"Action blocked: {e.violations}")

Supported frameworks: Google ADK, AutoGen, LangChain

Observability

Every classifier invocation is tracked automatically:

# After running checks
metrics = guard.metrics

# Get a summary
print(metrics.summary())
# {
#   "total_invocations": 42,
#   "by_classifier": {"llm:gpt-4o-mini": 23, "rule:basic": 19},
#   "avg_latency_ms": 145.2,
#   "confident_rate": 0.88,
#   "timeout_rate": 0.02,
# }

# Get raw invocations for detailed analysis
for invocation in metrics.to_list():
    print(f"{invocation.classifier_id}: {invocation.latency_ms}ms, confidence={invocation.confidence}")

Performance

Scenario	Latency
Clear violation caught by rules	<1ms
Action evaluated by local PDP model	~10-30ms
Avg across 1000 queries (cascade)	<50ms
Speculative layer (concurrent)	~max(fastest, all_uncertain)

---

Local PDP model — saroku-guard

saroku-guard protects every SafetyGuard() by default — no setup, no API key, no data leaving your environment. It downloads automatically on first use and runs on CPU.

guard = SafetyGuard()  # saroku-guard is already active

To use a different checkpoint, or disable it in favor of an LLM-only judge:

guard = SafetyGuard(local_model_path="your-org/your-model")
guard = SafetyGuard(use_local_pdp=False, judge_model="gpt-4o-mini")

Train your own

If you want to fine-tune on your own data or domain:

pip install trikesh[train]
python -m trikesh.training.trainer --output-dir ./my-model --epochs 3

---

Result object

result = guard.check(...)

result.is_safe          # bool
result.violations       # list of SafetyViolation
result.latency_ms       # float
result.layers_used      # ["rules", "ml", "local_model"]
result.ml_risk_score    # float 0-1
result.summary()        # human-readable string

Each SafetyViolation:

v.property        # "trust_hierarchy", "minimal_footprint", etc.
v.severity        # "high", "medium", "low"
v.description     # what the violation is
v.recommendation  # what to do instead
v.source          # "rules", "ml", or "local_model"

---

License

MIT