TrustyAI Garak: LLM Red Teaming for Eval-Hub

Automated vulnerability scanning and red teaming for Large Language Models using Garak. This package provides a FrameworkAdapter for the eval-hub evaluation platform, enabling garak scans to run as Kubernetes Jobs or via Kubeflow Pipelines.

What It Does

• 🔍 Vulnerability Assessment: Red-team LLMs for prompt injection, jailbreaks, toxicity, bias and other vulnerabilities
• 📋 Compliance: OWASP LLM Top 10, AVID taxonomy benchmarks
• 🎯 Intents-based Testing: Policy taxonomy + SDG + TAPIntent for targeted risk assessment (KFP mode)
• ☁️ Cloud-Native: Runs on OpenShift AI / Kubernetes as eval-hub jobs
• 📊 Detailed Reports: JSONL, HTML, and AVID-format reports with MLflow integration

Execution Modes

Installation

# Core (eval-hub adapter with KFP support)
pip install llama-stack-provider-trustyai-garak

# With SDG support (for intents workflows)
pip install "llama-stack-provider-trustyai-garak[sdg]"

# Development
pip install "llama-stack-provider-trustyai-garak[dev]"

Container Image

# Build the eval-hub adapter image
docker build -f Containerfile -t trustyai-garak:dev .

The container runs as:

# Simple mode (garak in same pod)
CMD ["python", "-m", "llama_stack_provider_trustyai_garak.evalhub"]

# KFP mode (garak in a separate KFP pod)
CMD ["python", "-m", "llama_stack_provider_trustyai_garak.evalhub.kfp_adapter"]

Benchmark Profiles

Predefined scan profiles available via benchmark_id:

Job Spec Configuration

The adapter reads a JobSpec from a mounted ConfigMap:

{
  "id": "scan-001",
  "provider_id": "garak",
  "benchmark_id": "quick",
  "model": {
    "url": "https://my-model-endpoint.example.com/v1",
    "name": "my-model"
  },
  "parameters": {
    "probes": "dan.Dan_11_0",
    "execution_mode": "simple"
  }
}

Parameters

Key parameters in the job spec:

Results

The adapter reports EvaluationResult per probe with:

• attack_success_rate: Percentage of successful attacks
• vulnerable_responses: Count of vulnerable responses
• total_attempts: Total probe attempts

Overall metrics include TBSA (Tier-Based Security Aggregate) when available.

Development

make test        # Run all tests
make coverage    # With coverage report
make lint        # ruff check
make format      # ruff format

Support & Documentation

• 📖 Garak Docs: https://reference.garak.ai/en/latest/index.html
• 💬 Issues: https://github.com/trustyai-explainability/llama-stack-provider-trustyai-garak/issues