q-uestionable-ai 0.10.8

Security testing for agentic AI

{q-AI}

Security testing for agentic AI

A research program testing whether MCP and agentic AI vulnerabilities are exploitable end-to-end, with execution-level proof.

Model-layer tests like Garak, PyRIT, or BIPIA can tell you "this model follows injected instructions." An audit scan can tell you "this MCP server has a vulnerable tool." A qai finding connects the two: the model weakness was exploitable end-to-end through a real agentic system. Authenticated callbacks confirm execution, not just compliance.

The Intel workspace is the operator surface. A target-centric view where scan results, probe runs, and sweep findings accrue per target and drive payload generation for the end-to-end tests.

Capabilities:

• Audit MCP servers
• Intercept agent traffic
• Test tool poisoning and prompt injection
• Execute multi-step attack chains
• Generate IPI payloads (adversarial documents with authenticated callbacks)
• Poison coding assistant context files
• Measure RAG retrieval rank

Local web UI orchestrates multi-module workflows. All findings stored in a SQLite database.

By Richard Spicer · {q-AI}

---

Quick Start

Scan an MCP server:

qai audit scan http://localhost:3000/sse

Transport is inferred automatically. Launch the web UI:

qai ui

---

Bring What You Have

Already running Garak or PyRIT, or working from BIPIA benchmarks? Import their results and let qai prove whether the weaknesses they found are exploitable in real agentic systems.

qai targets add "My Server" http://localhost:3000/sse
qai import report.jsonl --format garak --target <target-id>
qai import conversations.json --format pyrit --target <target-id>
qai import bipia.csv --format bipia --target <target-id>

Imported findings drive qai's native modules — inject payloads are prioritized based on compliance patterns your tools already discovered. Also supports SARIF from any tool.

---

Built-in Assistant

An AI assistant helps you discover capabilities, interpret scan results, and plan testing workflows. It uses RAG over qai's documentation and your own reference material, with a trust boundary model that separates trusted docs from untrusted scan output. Works with local models (Ollama) or cloud APIs.

qai config set assist.provider ollama
qai config set assist.model llama3.1
qai assist "how do I scan an MCP server?"

---

Framework Coverage

All audit findings map to four security taxonomies:

Framework	Coverage
OWASP MCP Top 10	All 10 categories
OWASP Agentic Top 10	All 10 categories
MITRE ATLAS	Technique-level mapping per finding category
CWE	Weakness-level mapping per finding category

---

Install

pip install q-uestionable-ai

Or from source:

git clone https://github.com/q-uestionable-AI/qai.git
cd qai
uv sync --group dev

---

Full documentation at docs.q-uestionable.ai

---

Legal

All tools are intended for authorized security testing only. Only test systems you own, control, or have explicit permission to test. Responsible disclosure for all vulnerabilities discovered.

License

Apache 2.0

AI Disclosure

This project uses a human-led, AI-augmented workflow. See AI-STATEMENT.md