AICAudit 🛡️

Evidence-driven AI code audit for Python — every verdict ships with a machine-checkable taint path.

证据链驱动的 Python 代码审计：每个判定都附带可复核的污点传播路径（source → … → sink）

Why AICAudit

Other tools give you a verdict. AICAudit gives you the evidence:

S001  app.py:16  SQL injection risk
  taint path: app.py:14 request.args (Flask user input)
            -> app.py:14 assigned to 'uid'
            -> app.py:15 assigned to 'query'
            -> app.py:16 conn.execute()

• Taint engine (pure Python, zero plugins): tracks user input from source (Flask/Django/FastAPI request, input(), os.environ, sys.argv) through assignments, f-strings, concatenation, and across function boundaries to the sink — and proves constants safe, killing the classic false-positive classes (open(BASE_DIR/"x"), constant SQL variables, eval("1+1"))
• AI verdicts with receipts: the LLM sees the full function and the taint path, not a one-line snippet — hallucination-resistant by construction
• SARIF codeFlows: taint paths render natively in the GitHub Security tab
• Zero config: pip install and scan; works offline, local LLMs supported

Quick Start

# From source (PyPI package arrives with v0.2.0)
pip install git+https://github.com/IhateStomachProblems/aicaudit.git

# Scan a file or directory
aicaudit scan ./src

# Markdown report (default)
aicaudit scan ./src --output markdown

# JSON output (for CI / scripts / AI agent integration)
aicaudit scan ./src --output json

# SARIF 2.1 output (GitHub Code Scanning compatible)
aicaudit scan ./src --output sarif

# Chinese language
aicaudit scan ./src --lang zh

# CI gate: exit 1 when any finding is error or worse (0 clean / 2 usage error)
aicaudit scan ./src --fail-on error

CI & Git Integration

One-line GitHub Action (this repo ships a composite action at its root):

- uses: IhateStomachProblems/aicaudit@main
  with:
    path: .
    fail-on: error      # optional: fail the job at this severity
    sarif: "true"       # optional: upload SARIF to Code Scanning

pre-commit (local hook until the PyPI release):

repos:
  - repo: local
    hooks:
      - id: aicaudit
        name: aicaudit
        entry: aicaudit scan --fail-on error
        language: system
        types: [python]

Project config — generate it interactively:

aicaudit init            # writes [tool.aicaudit] into pyproject.toml

Custom Rules (Python files, no DSL)

Drop rule files into .aicaudit/rules/ (auto-discovered) or list extra dirs under [tool.aicaudit] rule-dirs. Rules are ordinary Python using the public API — a later registration with the same id overrides the builtin:

# .aicaudit/rules/no_print.py
import ast
from aicaudit.rules.base import Finding, Rule, Severity, register

@register
class NoPrint(Rule):
    id = "Q100"
    name = "no-print"
    severity = Severity.INFO
    description = "Detect print() calls"

    def check(self, tree, context):
        return [Finding(rule_id=self.id, message="print() — prefer logging",
                        message_zh="print()——建议改用 logging",
                        file=str(context.file_path), line=n.lineno,
                        severity=self.severity)
                for n in ast.walk(tree)
                if isinstance(n, ast.Call) and isinstance(n.func, ast.Name)
                and n.func.id == "print"]

A ready-to-copy example lives at examples/custom_rule_example.py. (A declarative YAML rule format is planned for v0.3 — the loader is ready.)

Rules

Security

Quality

Performance

Inline Suppression

Suppress specific findings with inline comments:

# Ignore a specific rule on this line
query = f"SELECT * FROM users WHERE id={user_id}"  # aicaudit: ignore S001

# Ignore all rules on this line
eval(user_input)  # aicaudit: ignore

Web UI

Prefer a browser over the terminal? Start the built-in web interface:

aicaudit web          # http://127.0.0.1:8080

The UI is vendored and offline-first — no CDN, no build step, works air-gapped:

• Results browser: code context with syntax highlighting (server-side pygments), the taint path as a visual stepper (source → hops → sink), AI verdict card with confidence, and diff-preview/apply/rollback for fixes
• Live scan progress: per-file streaming (SSE), no fake progress bars
• Scan history: sessions persist under .aicaudit/web/, dashboard shows trends across runs
• Rules browser and AI provider config (relay / OpenAI / Claude / OpenRouter / local ollama)

API docs at /docs (Swagger UI). Found something rough? Open an issue.

GitHub Code Scanning Integration

AICAudit produces SARIF 2.1 output compatible with GitHub Code Scanning:

aicaudit scan ./src --output sarif > aicaudit.sarif

Upload the result in a GitHub Actions workflow:

- name: Run AICAudit
  run: aicaudit scan . --output sarif > aicaudit.sarif

- name: Upload SARIF
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: aicaudit.sarif

Caveats

AICAudit is a young project. Here are some honest limitations:

• Python only for now — other languages are planned
• Static analysis — not a runtime security tool
• Best-effort — no tool catches every bug ; please review findings critically
• The rules reflect common patterns but may not fit every codebase

Performance

Benchmarks

Labeled corpus (104 cases: function-level samples + a realistic Flask app with planted vulnerabilities), scored against ground truth with an honest methodology — including the categories where aicaudit is deliberately conservative or deliberately silent. Bandit runs on the same corpus for reference; buckets Bandit does not cover are marked.

Full table with F1, timing, and the honest-notes section (design tiers, conservative flags): benchmarks/RESULTS.md. Reproduce and challenge it yourself:

python benchmarks/run.py

Testing

• 305 unit tests (pytest)
• 92% code coverage (pytest-cov)
• Taint engine: 17 dedicated tests (sources, constness, sanitizers, interprocedural)
• AI pipeline: fail-safe parsing, transport retry, category-aware prompts all tested
• Integration tests for CLI, JSON, Markdown, SARIF, Web UI, Chinese output
• Self-scan validation: we audit our own codebase
• CI: GitHub Actions on Python 3.10–3.13 (ruff + mypy + coverage + self-scan)

AI Verdicts (evidence-grounded, fail-safe)

aicaudit scan ./src --ai attaches a verdict to every finding. The LLM sees the full enclosing function, the file's imports, and the engine-traced taint path — not a one-line snippet — and answers with a structured verdict (is_real, confidence, cwe, reason, suggested_fix).

Three verdict states, by design:

Why not auto-filter? The best published LLM-verifier configurations still wrongly suppress ~22% of true vulnerabilities (arXiv 2601.22952), so verdicts mark findings instead of deleting them. If you want aggressive filtering anyway, it is an explicit opt-in:

aicaudit scan ./src --ai --ai-strict    # keep only AI-confirmed findings

Prompts are category-aware: injection-class findings (S001/S003/S004/S005/S007) get dataflow verification instructions anchored on the taint path; policy-class findings (S002/S006/S008) get usage-context instructions — targeting the documented failure modes of LLM verifiers (surface pattern matching, CWE mislabeling, crypto/policy dismissals).

AI Configuration

AICAudit supports multiple LLM providers for AI-powered verification.

Direct API

# OpenAI
export AICAUDIT_AI_PROVIDER=openai
export AICAUDIT_AI_KEY=sk-xxx
aicaudit scan ./src --ai

# Claude
export AICAUDIT_AI_PROVIDER=claude
export ANTHROPIC_API_KEY=sk-ant-xxx
aicaudit scan ./src --ai

Relay / Proxy Service (中转接口)

Any OpenAI-compatible relay service works. Set the provider to relay and point to your relay endpoint:

# Example: API2D, OhMyGPT, NewAPI, OneAPI, etc.
export AICAUDIT_AI_PROVIDER=relay
export AICAUDIT_AI_BASE=https://your-relay.com/v1
export AICAUDIT_AI_KEY=sk-your-key
export AICAUDIT_AI_MODEL=gpt-4o-mini
aicaudit scan ./src --ai

Also accepts custom or proxy as provider names for the same behavior.

Local Models

export AICAUDIT_AI_PROVIDER=ollama
aicaudit scan ./src --ai

License

MIT © IhateStomachProblems

# 源码安装（v0.2.0 将上架 PyPI）
pip install git+https://github.com/IhateStomachProblems/aicaudit.git
aicaudit scan ./项目目录    # 扫描项目
aicaudit scan ./src --lang zh  # 使用中文输出
aicaudit scan ./src --output json  # JSON 输出
aicaudit scan ./src --output sarif  # SARIF 输出（GitHub Code Scanning 兼容）

# 忽略特定规则
query = f"SELECT * FROM users WHERE id={user_id}"  # aicaudit: ignore S001

# 忽略该行所有规则
eval(user_input)  # aicaudit: ignore

aicaudit scan ./src --output sarif > aicaudit.sarif

- name: Run AICAudit
  run: aicaudit scan . --output sarif > aicaudit.sarif

- name: Upload SARIF
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: aicaudit.sarif