MCP Server For Garak LLM Vulnerability Scanner
A lightweight MCP (Model Context Protocol) server for Garak.
Example:
https://github.com/user-attachments/assets/f6095d26-2b79-4ef7-a889-fd6be27bbbda
Tools Provided
Overview
| Name | Description |
|---|---|
| list_model_types | List all available model types (ollama, openai, huggingface, ggml) |
| list_models | List all available models for a given model type |
| list_garak_probes | List all available Garak attacks/probes |
| get_report | Get the report of the last run |
| run_attack | Run an attack with a given model and probe |
Detailed Description
-
list_model_types
- List all available model types that can be used for attacks
- Returns a list of supported model types (ollama, openai, huggingface, ggml)
-
list_models
- List all available models for a given model type
- Input parameters:
model_type(string, required): The type of model to list (ollama, openai, huggingface, ggml)
- Returns a list of available models for the specified type
-
list_garak_probes
- List all available Garak attacks/probes
- Returns a list of available probes/attacks that can be run
-
get_report
- Get the report of the last run
- Returns the path to the report file
-
run_attack
- Run an attack with the given model and probe
- Input parameters:
model_type(string, required): The type of model to usemodel_name(string, required): The name of the model to useprobe_name(string, required): The name of the attack/probe to use
- Returns a list of vulnerabilities found
Prerequisites
-
Python 3.11 or higher: This project requires Python 3.11 or newer.
# Check your Python version python --version
-
Install uv: A fast Python package installer and resolver.
pip install uv
Or use Homebrew:
brew install uv
-
Optional: Ollama: If you want to run attacks on ollama models be sure that the ollama server is running.
ollama serve
Installation
- Clone this repository:
git clone https://github.com/BIGdeadLock/Garak-MCP.git
- Configure your MCP Host (Claude Desktop ,Cursor, etc):
{
"mcpServers": {
"garak-mcp": {
"command": "uv",
"args": ["--directory", "path-to/Garak-MCP", "run", "garak-server"],
"env": {}
}
}
}
Tested on:
- Cursor
- Claude Desktop
Running Vulnerability Scans
You can run Garak vulnerability scans directly using the included CLI tool.
Prerequisites for Scanning
-
Ollama must be running:
ollama serve -
Pull a model to scan:
ollama pull llama2
Using the CLI Scanner
After installation, you can use the garak-scan command:
# List available Ollama models
uv run garak-scan --list-models
# Scan a specific model with all probes
uv run garak-scan --model llama2
# Scan with specific probes
uv run garak-scan --model llama2 --probes encoding
# Scan with custom output directory
uv run garak-scan --model llama2 --output-dir ./my_scans
# Run multiple parallel attempts
uv run garak-scan --model llama2 --parallel-attempts 4
Scan Results
Scan results are saved in the output/ directory (or your specified directory) as JSONL files. Each scan creates a timestamped report file:
output/scan_llama2_20250125_143022.report.jsonl
GitHub Actions Integration
This repository includes a GitHub Actions workflow that automatically runs vulnerability scans:
- Triggers: Push to main/master, pull requests, weekly schedule (Mondays at 2am UTC)
- Manual runs: Go to Actions → Garak LLM Vulnerability Scan → Run workflow
- Custom options: Specify model and probes when running manually
- Results: Scan results are uploaded as workflow artifacts
To enable automated scanning:
- Ensure the workflow file exists at
.github/workflows/garak-scan.yml - Push to your repository
- Check the Actions tab to view scan results
Future Steps
- Add support for Smithery AI: Docker and config
- Improve Reporting
- Test and validate OpenAI models (GPT-3.5, GPT-4)
- Test and validate HuggingFace models
- Test and validate local GGML models
Metadata
Release files for iflow-mcp_edenyavin-garakr-mcp 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| iflow_mcp_edenyavin_garakr_mcp-0.1.0.tar.gz | 8.7 kB | Without provenance | Details for iflow_mcp_edenyavin_garakr_mcp-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| iflow_mcp_edenyavin_garakr_mcp-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for iflow_mcp_edenyavin_garakr_mcp-0.1.0-py3-none-any.whl |
Total release size: 19.8 kB
Release files / iflow_mcp_edenyavin_garakr_mcp-0.1.0.tar.gz
| Download URL | iflow_mcp_edenyavin_garakr_mcp-0.1.0.tar.gz |
|---|---|
| Size | 8.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
07423cb5220b0985da017426b18a9fcaeedfd284bd1590bb0284ced351dfbbb3
|
|
BLAKE2b-256 checksum How to use checksums |
ed802be184028ec6d4a543aaee992a4a03315c2c787d479146f751326d5467e3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.10.2 {"installer":{"name":"uv","version":"0.10.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Debian GNU/Linux","version":"13","id":"trixie","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / iflow_mcp_edenyavin_garakr_mcp-0.1.0-py3-none-any.whl
| Download URL | iflow_mcp_edenyavin_garakr_mcp-0.1.0-py3-none-any.whl |
|---|---|
| Size | 11.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9c29d792f64ebc5e1e49a3892b01d555dad8c9ad34ed51a5edc382a71c2c3108
|
|
BLAKE2b-256 checksum How to use checksums |
96ae47595becf04e055e072a2be91d1a4dba01a5da48c476134abe5fd4421c9b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.10.2 {"installer":{"name":"uv","version":"0.10.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Debian GNU/Linux","version":"13","id":"trixie","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for iflow-mcp_edenyavin-garakr-mcp 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.