promptstrike 0.1.0

Developer-first automated LLM red-team platform

🎯 PromptStrike CLI

Developer-first automated LLM red-team platform

🚀 Status: ✅ Sprint S-1 Complete (July 2025) - Production Ready
📋 Reference: Product One-Pager | 12M Roadmap | Changelog

Problem We Solve

Large-language-model (LLM) apps ship to production with invisible jailbreak, data-leak and cost-explosion risks. Regulators now mandate continuous red-teaming (EU AI Act Art.55, US EO 14110).

PromptStrike provides automated OWASP LLM Top 10 testing that:

• ✅ Runs locally (keys stay on-prem)
• ✅ Generates audit-ready evidence (NIST AI-RMF mapping)
• ✅ Ships as Docker CLI (5-minute setup)
• ✅ Tracks coverage and compliance gaps

Quick Start

🐳 Docker (Recommended)

# Pull and run latest version
docker run --rm \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  -v $(pwd)/reports:/app/reports \
  promptstrike/cli:latest scan gpt-4

# Or build locally
git clone https://github.com/siwenwang0803/PromptStrike.git
cd PromptStrike
docker build -t promptstrike/cli .
docker run --rm \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  -v $(pwd)/reports:/app/reports \
  promptstrike/cli scan gpt-4 --format pdf

📦 Poetry (Development)

git clone https://github.com/siwenwang0803/PromptStrike.git
cd PromptStrike
poetry install
poetry run promptstrike scan gpt-4 --dry-run

🔧 PyPI (Future - Sprint S-4)

pip install promptstrike
promptstrike scan gpt-4

Core Features

🎯 Automated LLM Red-Team Testing

• OWASP LLM Top 10 complete coverage (47 attack patterns)
• Prompt injection, data leakage, cost exploitation detection
• Confidence scoring and risk assessment (CVSS-like 0-10 scale)
• Real-time progress with rich CLI interface

📊 Comprehensive Reporting

• JSON structured reports for CI/CD integration
• PDF executive summaries for compliance teams
• HTML interactive dashboards for security teams
• CSV exports for spreadsheet analysis

🏛️ Compliance & Audit Ready

• NIST AI-RMF control mapping (GV, MP, MS, MG categories)
• EU AI Act article references (Art.15, 16, 52, 55)
• SOC 2 impact analysis (CC6.1, CC6.7, CC7.2, CC8.1)
• Cryptographic audit trails with evidence preservation

🚀 Production Ready

• Docker containerized for consistent deployment
• Rate limiting and timeout controls
• Parallel execution for faster scans
• Error handling and graceful degradation

Command Reference

Basic Usage

# Quick vulnerability scan
promptstrike scan gpt-4

# Comprehensive scan with PDF report
promptstrike scan https://api.openai.com/v1/chat/completions \
  --format pdf --output ./security-audit

# Preview attacks without execution
promptstrike scan local-model --dry-run

# List available attack packs
promptstrike list-attacks

# Health check
promptstrike doctor

Advanced Usage

# Custom configuration file
promptstrike scan gpt-4 --config ./config.yaml --verbose

# Limited test run for CI/CD
promptstrike scan $MODEL --max-requests 20 --timeout 10

# Multiple output formats
promptstrike scan gpt-4 --format all --output ./reports

Attack Packs

🔴 OWASP LLM Top 10 (Default)

Category	Attacks	Severity	Description
LLM01 - Prompt Injection	12	Critical	Direct/indirect prompt manipulation
LLM02 - Insecure Output	6	High	XSS, code injection via outputs
LLM03 - Training Data Poisoning	4	Medium	Backdoor and bias injection
LLM04 - Model DoS	8	High	Resource exhaustion attacks
LLM05 - Supply Chain	3	Medium	Third-party model vulnerabilities
LLM06 - Sensitive Info Disclosure	7	Critical	PII and secret extraction
LLM07 - Insecure Plugin Design	5	High	Plugin-specific vulnerabilities
LLM08 - Excessive Agency	4	Medium	Overprivileged model actions
LLM09 - Overreliance	3	Low	Human oversight failures
LLM10 - Model Theft	2	Medium	IP extraction attempts

🔮 Coming Soon (Roadmap)

• PromptStrike FinOps - Cost explosion & billing anomaly detection (Sprint S-9)
• PromptStrike Privacy - GDPR/CCPA compliance validation (Sprint S-10)
• PromptStrike Bias - Fairness and discrimination testing (Sprint S-11)

Report Format

JSON Schema (API Integration)

{
  "scan_id": "ps-20250703-140502-abc123",
  "target": "gpt-4",
  "overall_risk_score": 6.7,
  "security_posture": "fair",
  "vulnerabilities_found": 3,
  "results": [
    {
      "attack_id": "LLM01-001",
      "category": "prompt_injection",
      "severity": "critical",
      "is_vulnerable": false,
      "confidence_score": 0.95,
      "risk_score": 2.1,
      "nist_controls": ["GV-1.1", "MP-2.3"],
      "eu_ai_act_refs": ["Art.15"]
    }
  ],
  "compliance": {
    "nist_rmf_controls_tested": ["GV-1.1", "MP-2.3"],
    "eu_ai_act_risk_category": "high",
    "soc2_controls_impact": ["CC6.1", "CC6.7"]
  }
}

PDF Executive Summary

1. Risk Overview - Security posture, vulnerability count by severity
2. Technical Findings - Detailed attack results with evidence
3. Compliance Assessment - NIST AI-RMF, EU AI Act, SOC 2 mapping
4. Remediation Plan - Prioritized recommendations and controls

Configuration

Create promptstrike.yaml for custom settings:

target:
  endpoint: "https://api.openai.com/v1/chat/completions"
  model: "gpt-4"
  api_key_env: "OPENAI_API_KEY"

scan:
  max_requests: 100
  timeout: 30
  parallel_workers: 3
  rate_limit_rps: 5

attack_packs:
  default: "owasp-llm-top10"
  enabled: ["owasp-llm-top10"]

output:
  directory: "./reports"
  formats: ["json", "pdf"]
  retention_days: 30

compliance:
  nist_rmf_enabled: true
  eu_ai_act_enabled: true
  soc2_enabled: false

🎯 Sprint S-1 Completion

Production-Ready Release (July 2025)

✅ Delivered Features

• Complete OWASP LLM Top 10: 47 attacks across 10 categories
• Multi-Format Reports: JSON, HTML, PDF with compliance mapping
• Docker Deployment: Production-ready containerized CLI
• CLI Interface: 5 commands with Rich terminal UI
• Compliance Ready: NIST AI-RMF, EU AI Act, SOC 2 integration
• Local Execution: Zero data exfiltration, keys stay on-premises
• CI/CD Integration: GitHub Actions, Jenkins pipeline support

📊 Technical Metrics

• 47 Attack Patterns: Complete coverage of OWASP LLM Top 10
• 5 CLI Commands: scan, list-attacks, doctor, version, config
• 3 Report Formats: JSON (API), HTML (web), PDF (executive)
• 100% Local: All operations run locally, no cloud dependencies
• Docker Ready: Multi-stage build with security best practices

🚀 Next: Pilot-0 / Sprint S-2

• Kubernetes Sidecar: Deployment to K8s clusters
• Python SDK: Programmatic access and integration
• Real-time Dashboard: Live monitoring and alerting
• Custom Attack Packs: User-defined vulnerability tests
• Advanced Compliance: Detailed gap analysis and remediation

🎯 Business Goals

• 500 Downloads: PyPI package adoption milestone
• 5 GitHub Issues: Community engagement and feedback
• Design Partners: Enterprise pilot customer onboarding

---

CI/CD Integration

GitHub Actions

- name: PromptStrike Security Scan
  run: |
    docker run --rm \
      -e OPENAI_API_KEY=${{ secrets.OPENAI_API_KEY }} \
      -v ${{ github.workspace }}/reports:/app/reports \
      promptstrike/cli:latest scan ${{ env.MODEL }} \
      --format json --max-requests 50
    
    # Fail build if critical vulnerabilities found
    if [ $? -eq 3 ]; then
      echo "Critical vulnerabilities detected - blocking deployment"
      exit 1
    fi

Jenkins Pipeline

stage('LLM Security Scan') {
    steps {
        script {
            def result = sh(
                script: """
                    docker run --rm \
                        -e OPENAI_API_KEY=\$OPENAI_API_KEY \
                        -v \$PWD/reports:/app/reports \
                        promptstrike/cli:latest scan \$MODEL
                """,
                returnStatus: true
            )
            if (result == 3) {
                error("Critical LLM vulnerabilities found")
            }
        }
    }
}

Development

Prerequisites

• Python 3.11+
• Poetry 1.7+
• Docker (optional)
• OpenAI API key

Setup

# Clone repository
git clone https://github.com/siwenwang0803/PromptStrike.git
cd PromptStrike

# Install dependencies
make install

# Run tests
make test

# Format code
make format

# Build Docker image
make docker-build

# Run CLI locally
make cli-help

Testing

# Run all tests
make test

# Fast tests only
make test-fast

# Specific test file
poetry run pytest tests/test_cli.py -v

# Coverage report
make test && open htmlcov/index.html

Development Commands

# Start development environment
make dev

# Health check
make cli-doctor

# Dry run example
make cli-dry-run

# Schema validation
make schema-export

Roadmap & Status

✅ Sprint S-1 (Jul 08-21) - SHIPPED

• Dockerized CLI with Poetry environment
• OWASP LLM Top 10 attack pack (47 attacks)
• JSON/PDF report generation
• NIST AI-RMF compliance mapping
• Target: 500 downloads, 5 GitHub issues closed

🚧 Sprint S-2 (Jul 22-Aug 04) - IN PROGRESS

• Guardrail Side-car α (Kubernetes deployment)
• Python SDK for programmatic access
• Real-time monitoring dashboard
• Target: Live in 1 design partner staging environment

📋 Upcoming Sprints

• S-3: Pilot template, Stripe checkout, $15k revenue target
• S-4: SaaS Dashboard α (Next.js + Supabase)
• S-5: NIST AI-RMF & EU AI-Act mapping β

See 12-Month Roadmap for complete timeline.

Support & Community

🐛 Bug Reports & Feature Requests

• GitHub Issues
• Security Vulnerabilities

📚 Documentation

• CLI Specification
• API Documentation (Coming Soon)
• Attack Pack Reference (Coming Soon)

💬 Community

• Discord Server (Coming Soon)
• Slack Channel (Design Partners)

Security & Privacy

• 🔒 API Keys: Never logged or transmitted beyond target endpoint
• 🏠 Local Execution: All scanning runs on your infrastructure
• 📊 Telemetry: Opt-in anonymous usage statistics only
• 🔍 Audit Trail: Cryptographic evidence preservation
• 🛡️ Responsible Disclosure: [e-mail]

License & Legal

MIT License - See LICENSE for details.

Disclaimer: PromptStrike is a security testing tool intended for authorized testing only. Users are responsible for compliance with applicable laws and regulations. The authors assume no liability for misuse.

Contributors

Core Team (AI Agents)

• Claude 4 Sonnet - Full-stack Development & Testing Lead
• ChatGPT o3-pro - Strategy VP & Principal Architect
• ChatGPT o3 - Sprint PM & Delivery Lead
• GPT-4.5 - Frontend & Documentation Lead
• gork - OTEL & Automation Engineer

Design Partners

• [Confidential] - 3 enterprise partners in pilot phase

Star History

⭐ Star this repo if PromptStrike helps secure your LLM applications!

---

🎯 Ready to secure your LLM? Start with: docker run promptstrike/cli scan gpt-4 --dry-run

📧 Questions? Reach out: [e-mail]

🚀 Want the SaaS version? Join our waitlist: promptstrike.com