Skip to main content

Detect security vulnerabilities in AI-generated code

Project description

๐Ÿ›ก๏ธ AI Code Guard

Detect security vulnerabilities in AI-generated code before they reach production

Python 3.8+ License: MIT Security Tool

AI coding assistants (GitHub Copilot, Claude, ChatGPT, Cursor) are revolutionizing development โ€” but they can introduce security vulnerabilities that slip past code review. AI Code Guard scans your codebase for security issues commonly found in AI-generated code.

๐ŸŽฏ What It Detects

Category Examples
Prompt Injection Risks User input in system prompts, unsafe template rendering
Hardcoded Secrets API keys, passwords, tokens in AI-suggested code
Insecure Code Patterns SQL injection, command injection, path traversal
Data Exfiltration Risks Suspicious outbound requests, data leakage patterns
Dependency Confusion Typosquatting packages, suspicious imports

๐Ÿš€ Quick Start

# Install
pip install ai-code-guard

# Scan a directory
ai-code-guard scan ./src

# Scan a single file
ai-code-guard scan ./src/api/chat.py

# Output as JSON
ai-code-guard scan ./src --format json

๐Ÿ“Š Example Output

$ ai-code-guard scan ./my-project

๐Ÿ” AI Code Guard v0.1.0
   Scanning 47 files...

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ CRITICAL: SQL Injection Vulnerability                               โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ File: src/db/queries.py, Line 42                                    โ”‚
โ”‚ Code: query = f"SELECT * FROM users WHERE id = {user_id}"          โ”‚
โ”‚                                                                     โ”‚
โ”‚ AI-generated code often uses f-strings for SQL queries.            โ”‚
โ”‚ Use parameterized queries instead.                                  โ”‚
โ”‚                                                                     โ”‚
โ”‚ โœ… Fix: cursor.execute("SELECT * FROM users WHERE id = ?", (id,))  โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ HIGH: Prompt Injection Risk                                         โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ File: src/api/chat.py, Line 23                                      โ”‚
โ”‚ Code: prompt = f"You are a helper. User says: {user_input}"        โ”‚
โ”‚                                                                     โ”‚
โ”‚ User input directly concatenated into LLM prompt.                   โ”‚
โ”‚ Attacker can inject malicious instructions.                         โ”‚
โ”‚                                                                     โ”‚
โ”‚ โœ… Fix: Sanitize input and use structured prompt templates          โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ HIGH: Hardcoded API Key                                             โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ File: src/config.py, Line 15                                        โ”‚
โ”‚ Code: api_key = "sk-proj-abc123..."                                 โ”‚
โ”‚                                                                     โ”‚
โ”‚ AI assistants often generate code with placeholder secrets          โ”‚
โ”‚ that developers forget to remove.                                   โ”‚
โ”‚                                                                     โ”‚
โ”‚ โœ… Fix: Use environment variables: os.environ.get("API_KEY")        โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
๐Ÿ“Š SUMMARY
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
   Files scanned:  47
   Issues found:   3
   
   ๐Ÿ”ด CRITICAL:    1
   ๐ŸŸ  HIGH:        2
   ๐ŸŸก MEDIUM:      0
   ๐Ÿ”ต LOW:         0
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”

๐Ÿ”ง Configuration

Create .ai-code-guard.yaml in your project root:

# Severity threshold (ignore issues below this level)
min_severity: medium

# Patterns to ignore
ignore:
  - "tests/*"
  - "*.test.py"
  - "examples/*"

# Specific rules to disable
disable_rules:
  - "SEC001"  # Hardcoded secrets (if using .env.example)

# Custom secret patterns to detect
custom_secrets:
  - pattern: "my-company-api-.*"
    name: "Company API Key"

๐Ÿ“‹ Rule Reference

Rule ID Category Description
SEC001 Secrets Hardcoded API keys, passwords, tokens
SEC002 Secrets AWS/GCP/Azure credentials in code
INJ001 Injection SQL injection via string formatting
INJ002 Injection Command injection via os.system/subprocess
INJ003 Injection Path traversal vulnerabilities
PRI001 Prompt Injection User input in LLM system prompts
PRI002 Prompt Injection Unsafe prompt template rendering
PRI003 Prompt Injection Missing input sanitization for LLM
DEP001 Dependencies Known typosquatting packages
DEP002 Dependencies Suspicious import patterns
EXF001 Data Exfiltration Outbound requests with sensitive data
EXF002 Data Exfiltration Base64 encoding of sensitive variables

๐Ÿ”Œ CI/CD Integration

GitHub Actions

name: Security Scan

on: [push, pull_request]

jobs:
  ai-code-guard:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.10'
      - run: pip install ai-code-guard
      - run: ai-code-guard scan ./src --format sarif > results.sarif
      - uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: results.sarif

Pre-commit Hook

# .pre-commit-config.yaml
repos:
  - repo: https://ThorneShadowbane/ai-code-guard
    rev: v0.1.0
    hooks:
      - id: ai-code-guard

๐Ÿง  Why AI-Generated Code Needs Special Attention

AI coding assistants are trained on vast amounts of code โ€” including insecure patterns. Common issues include:

  1. Outdated Security Practices: Training data includes old, insecure code
  2. Placeholder Secrets: AI generates realistic-looking API keys as examples
  3. Prompt Injection Blindspots: Most training data predates LLM security concerns
  4. Context-Free Suggestions: AI doesn't understand your security requirements

This tool specifically targets patterns commonly introduced by AI assistants.

๐Ÿค Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines.

Adding New Detection Patterns

# ai_code_guard/patterns/my_pattern.py
from ai_code_guard.patterns.base import BasePattern, Finding, Severity

class MyCustomPattern(BasePattern):
    """Detect my custom security issue."""
    
    rule_id = "CUS001"
    name = "Custom Security Issue"
    severity = Severity.HIGH
    
    def scan(self, content: str, filepath: str) -> list[Finding]:
        findings = []
        # Your detection logic here
        return findings

๐Ÿ“š Research Background

This tool implements patterns identified in research on AI coding assistant security vulnerabilities. Key references:

๐Ÿ“„ License

MIT License โ€” see LICENSE for details.

๐Ÿ™ Acknowledgments

  • Security patterns informed by OWASP guidelines
  • Prompt injection research from the AI security community
  • Inspired by tools like Semgrep, Bandit, and GitLeaks

Built with ๐Ÿ›ก๏ธ by security engineers who use AI coding assistants daily

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ai_code_guard-0.1.1.tar.gz (25.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ai_code_guard-0.1.1-py3-none-any.whl (26.3 kB view details)

Uploaded Python 3

File details

Details for the file ai_code_guard-0.1.1.tar.gz.

File metadata

  • Download URL: ai_code_guard-0.1.1.tar.gz
  • Upload date:
  • Size: 25.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for ai_code_guard-0.1.1.tar.gz
Algorithm Hash digest
SHA256 d947fa5b7b359663931db79125a7ee1bb22258c45237ce7ba8f4637f93d18c9b
MD5 1dcc8b3df9bf796a95e6a9ddea4eaadd
BLAKE2b-256 0746c58affa99ff617e465e41036126c4fabcefe72e1fdf675573f278742e621

See more details on using hashes here.

File details

Details for the file ai_code_guard-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: ai_code_guard-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 26.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for ai_code_guard-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 7e43ce306c093b91c4f1327e4e5d971b48fb61b89c0585d014f1d51fb11991db
MD5 1e7ac535e301d1ccd5a68dda71734e73
BLAKE2b-256 3cdd84716774fe5909bb3cc1b3e66aa3d683d713326b5f562d7990297cacaf6c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page