LLMSafe

LLMSafe is an open-source static security scanner for AI-powered and agentic Python applications. It traces user input and model-controlled data into dangerous capabilities such as code execution, shells, SQL, outbound requests, and dynamic tool dispatch.

It runs locally. Source code is not uploaded to a model or external analysis service.

Status: v0.2.1 is an early release. LLMSafe provides reviewable security signals, not a guarantee that an AI system is secure.

Why another security scanner?

Traditional Python scanners are good at finding dangerous APIs. Agentic applications add a different question: can untrusted user or model output reach that capability?

flowchart LR
    A["User input"] --> C["Assignments and transforms"]
    B["Model output"] --> C
    C --> D["Shell / eval / SQL / HTTP / tool dispatch"]
    D --> E["Finding with source-to-sink evidence"]

LLMSafe combines focused API checks with AST-based dataflow and agent-framework rules:

def run_agent(client, user_input):
    response = client.responses.create(input=user_input)
    generated_code = response.output_text
    return eval(generated_code)

The scanner reports both the dangerous eval() and the path from the model response to that sink:

agent.py:4:12: CRITICAL FLOW001 Untrusted data reaches code execution
  Untrusted or model-controlled data flows into eval(). Source: model, user.
  Trace 2:16: model source: client.responses.create
  Trace 1:23: user source: user_input
  Trace 4:12: reaches eval
  Fix: Replace dynamic execution with a typed parser and an allow-listed operation.

Detection coverage

See the complete rule catalog and threat model.

Integrations can query the same catalog without parsing documentation:

llmsafe --list-rules
llmsafe --list-rules --format json

The versioned JSON output includes every stable ID, severity, family, description, and remediation.

Install

LLMSafe supports Python 3.9 and newer.

python3 -m venv .venv
source .venv/bin/activate
python -m pip install llmsafe

For development:

git clone https://github.com/rezerpaul-crypto/llmsafe.git
cd llmsafe
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev]"

Use the CLI

Scan the current repository:

llmsafe .

Scan selected paths and fail on medium-or-higher findings:

llmsafe src agent.py --fail-on medium --exclude "generated/**"

Generate machine-readable reports:

llmsafe . --format json --output reports/llmsafe.json
llmsafe . --format sarif --output reports/llmsafe.sarif

Exit codes are stable for automation:

Repository policy

Commit a .llmsafe.toml file:

[llmsafe]
exclude = ["generated/**", "vendor/**"]
fail_on = "high"
max_file_size = 1000000
disabled_rules = ["PY004"]

CLI options override or extend repository policy. Policy can also live under [tool.llmsafe] in pyproject.toml. See configuration.

Adopt LLMSafe without ignoring new risk

Existing repositories can review and commit a baseline of current findings:

llmsafe . --write-baseline .llmsafe-baseline.json
llmsafe . --baseline .llmsafe-baseline.json

The second command reports and fails only on findings not represented in the baseline. Matching is line-independent, duplicate-aware, and deterministic so ordinary code movement does not create noise while an additional dangerous operation is still reported. Baselines are review artifacts, not permanent suppressions; see incremental adoption.

Suppress one reviewed finding

Place a narrow suppression on the finding line or immediately above it:

# llmsafe: ignore[PY001] -- expression is generated from a fixed internal grammar
result = eval(TRUSTED_EXPRESSION)

Prefer a rule-specific suppression over a bare llmsafe: ignore.

GitHub Code Scanning

The repository includes a reusable composite action. A consumer workflow can scan, upload SARIF, then enforce the configured threshold:

permissions:
  contents: read
  security-events: write

steps:
  - uses: actions/checkout@v7
  - uses: actions/setup-python@v7
    with:
      python-version: "3.12"
  - id: llmsafe
    continue-on-error: true
    uses: rezerpaul-crypto/llmsafe@v0.2.1
    with:
      path: .
      fail-on: high
  - if: always()
    uses: github/codeql-action/upload-sarif@v4
    with:
      sarif_file: ${{ steps.llmsafe.outputs.sarif-file }}
  - if: steps.llmsafe.outcome == 'failure'
    run: exit 1

The workflow uses only contents: read and security-events: write.

Pre-commit

repos:
  - repo: https://github.com/rezerpaul-crypto/llmsafe
    rev: v0.2.1
    hooks:
      - id: llmsafe

Benchmark

The checked-in benchmark exercises vulnerable and safe agent boundaries:

python -m benchmarks.run

Current expectations cover 18 rule-level signals across direct and local-helper code execution, shell execution, SQL, SSRF, tool dispatch, prompt boundaries, high-impact tools, approval bypasses, and MCP. This is a regression corpus—not an industry benchmark or a claim of real-world detection rate. See the benchmark methodology.

How LLMSafe fits

Read the architecture for implementation boundaries and tradeoffs.

Contributing and security

Contributions are welcome. Start with CONTRIBUTING.md and the public roadmap. Report vulnerabilities privately according to SECURITY.md.

LLMSafe is released under the MIT License.