opspilot-ai 0.1.2

AI-powered production incident analysis tool using multi-agent systems

OpsPilot-AI 🤖

An intelligent agentic AI CLI tool for automated incident analysis and error resolution

OpsPilot-AI is your AI-powered Site Reliability Engineer that analyzes your projects, identifies runtime issues, and suggests safe fixes—all through a simple command-line interface.

---

🎯 What is OpsPilot-AI?

OpsPilot-AI uses a multi-agent AI architecture to understand your project's context, form hypotheses about runtime issues, and provide evidence-based fix recommendations. Think of it as having an experienced SRE on your team, available 24/7.

Key Capabilities

• 🔍 Intelligent Context Gathering - Automatically analyzes logs, environment variables, Docker configs, dependencies, and project structure
• 🧠 Multi-Agent Architecture - 4 specialized agents (Planner, Verifier, Fixer, Remediation) working collaboratively
• 🌐 Multi-Provider LLM Support - Automatic fallback across Ollama, OpenRouter, Gemini, and HuggingFace
• ✅ Evidence-Based Verification - Validates hypotheses with collected evidence and confidence scoring
• 🛠️ Safe Fix Suggestions - Provides dry-run suggestions with detailed rationale (never auto-applies changes)
• 💾 Redis-Based Memory - Auto-expiring incident history with user isolation and sub-second lookups
• 🚨 Severity Classification - Automatic P0/P1/P2/P3 incident prioritization
• ☁️ Production Log Fetching - S3, Kubernetes, CloudWatch, and HTTP endpoint support
• 📊 Deployment Correlation - Links incidents to recent Git deployments for faster root cause analysis

---

🚀 Quick Start (2 Minutes Setup)

Step 1: Install OpsPilot-AI

pip install opspilot-ai

Step 2: Setup LLM (Choose ONE Option)

OpsPilot-AI needs an LLM to analyze your code. Choose one of these options:

---

Option A: Ollama (Recommended - Free & Private)

Ollama runs locally on your machine. Your code never leaves your computer.

For macOS/Linux:

# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# Pull the AI model (one-time download, ~4GB)
ollama pull llama3

# Verify it's running
ollama list

For Windows:

1. Download Ollama from ollama.ai/download
2. Install and run the application
3. Open terminal and run:

ollama pull llama3

---

Option B: Cloud API (No Local Install Required)

Use cloud-based LLMs with free tiers. Set one of these environment variables:

Google Gemini (Recommended Cloud Option):

# Get free API key: https://makersuite.google.com/app/apikey
export GOOGLE_API_KEY="your-api-key-here"

OpenRouter (100+ Models Available):

# Get free API key: https://openrouter.ai/keys
export OPENROUTER_API_KEY="your-api-key-here"

HuggingFace:

# Get free token: https://huggingface.co/settings/tokens
export HUGGINGFACE_API_KEY="your-api-key-here"

Windows Users (set environment variable):

set GOOGLE_API_KEY=your-api-key-here

---

Step 3: Analyze Your Project

# Navigate to your project
cd /path/to/your/project

# Run analysis
opspilot analyze

That's it! OpsPilot-AI will analyze your project and provide diagnosis.

---

📖 Usage Examples

Basic Commands

# Quick analysis (fastest)
opspilot analyze --mode quick

# Deep analysis (thorough, recommended)
opspilot analyze --mode deep

# Verbose output (see what's happening)
opspilot analyze --verbose

Advanced Commands

# Analyze with production logs from S3
opspilot analyze --log-source s3://my-bucket/logs/app.log

# Analyze with deployment correlation (links errors to git commits)
opspilot analyze --deployment-analysis --since-hours 48

# JSON output for CI/CD automation
opspilot analyze --json --mode quick

# Analyze specific log file
opspilot analyze --log-source /var/log/myapp/error.log

# Full debugging output
opspilot analyze --verbose --debug

Analysis Modes

Mode	Speed	LLM Calls	Use Case
quick	Fast	1	Quick check, CI/CD pipelines
deep	Thorough	Up to 4	Detailed incident analysis
explain	Instant	0	Context gathering only (no LLM)

Installation Options

# Basic (just the CLI)
pip install opspilot-ai

# With Redis support (remembers past incidents)
pip install "opspilot-ai[redis]"

# With AWS support (S3, CloudWatch logs)
pip install "opspilot-ai[aws]"

# With Kubernetes support (K8s pod logs)
pip install "opspilot-ai[k8s]"

# Everything included
pip install "opspilot-ai[all]"

---

🔧 Troubleshooting

"No LLM providers available"

You need to set up an LLM. Choose one:

# Option 1: Install Ollama (recommended)
curl -fsSL https://ollama.ai/install.sh | sh
ollama pull llama3

# Option 2: Set a cloud API key
export GOOGLE_API_KEY="your-key"

"Ollama connection refused"

Make sure Ollama is running:

# Start Ollama service
ollama serve

# In another terminal, verify it works
ollama list

"No logs found"

OpsPilot-AI looks for logs in these locations:

• ./logs/ directory
• *.log files in project root
• Files specified with --log-source

# Specify log file directly
opspilot analyze --log-source ./my-app/error.log

"Context collected: 0 env vars"

Make sure you have a .env file in your project directory, or environment variables set.

---

Example Output:

Similar issues detected from past runs:
- Redis connection issue caused by network or Redis server downtime (confidence 0.8)

OpsPilot-AI initialized
Project detected: /your/project

Planner Agent reasoning...
Hypothesis: Redis connection issue
Confidence: 0.9

Evidence collected:
{'log_errors': {'ERROR': 1, 'Timeout': 1}, 'uses_redis': True}

Verifying hypothesis...
Supported: True
Confidence: 0.8
Reason: The presence of Redis connection-related errors (Timeout) and the system's use of Redis support the hypothesis.

Generating safe fix suggestions (dry-run)...

File: .env
Increase Redis timeout to reduce transient timeout errors under load.
--- a/.env
+++ b/.env
@@
-REDIS_TIMEOUT=1
+REDIS_TIMEOUT=5

File: app/config/redis.py
Enable connection pooling and reasonable timeouts to improve reliability.
--- a/app/config/redis.py
+++ b/app/config/redis.py
@@
-redis.Redis(host=host, port=port)
+redis.Redis(host=host, port=port, socket_timeout=5, max_connections=20)

---

🏗️ Architecture

OpsPilot-AI implements a multi-agent agentic architecture with four specialized agents:

1. Planner Agent - Analyzes project context and forms hypotheses about root causes
2. Verifier Agent - Collects evidence and validates hypotheses with confidence scoring
3. Fixer Agent - Generates safe, actionable fix suggestions
4. Remediation Agent - Creates 3-tier action plans (immediate, short-term, long-term)

Multi-Provider LLM System:

• Automatic fallback routing across 4 providers
• Connection pooling for high availability
• Provider health metrics and monitoring

Redis-Based Memory:

• User-isolated incident storage with SHA-256 project hashing
• Automatic TTL expiration (configurable, default 30 days)
• Sub-second similarity search with sorted sets
• Severity-based indexing (P0/P1/P2/P3)

See ARCHITECTURE.md for detailed design documentation.

---

🧩 How It Works

1. Context Collection

OpsPilot-AI gathers information from multiple sources:

• Logs: Recent error logs and exceptions
• Environment: Environment variables and configurations
• Dependencies: Project dependencies (requirements.txt, package.json)
• Docker: Dockerfile and docker-compose configurations
• Structure: Project file tree and organization

2. Hypothesis Generation

The Planner agent uses LLM reasoning to:

• Analyze collected context
• Identify patterns and anomalies
• Form hypotheses about root causes
• Assign confidence scores (0.0 - 1.0)

3. Evidence-Based Verification

The Verifier agent:

• Collects concrete evidence (log errors, missing configs, etc.)
• Cross-references with the hypothesis
• Updates confidence based on evidence strength
• Provides reasoning for the verdict

4. Safe Fix Suggestions

If confidence ≥ 0.6, the Fixer agent:

• Generates actionable fix suggestions as diffs
• Explains the rationale for each fix
• Provides domain-specific solutions (e.g., Redis timeout fixes)
• Never auto-applies changes (dry-run only for safety)

5. Learning from History

OpsPilot-AI maintains Redis-based memory of past issues:

• Stores hypotheses, confidence scores, and evidence with automatic TTL
• User-isolated storage using project path hashing
• Detects similar issues in future runs with sub-second lookups
• Automatic expiration prevents stale incident data
• Falls back to file-based storage if Redis unavailable

---

🎓 Technology Stack

• LLM Integration: Multi-provider system (Ollama, OpenRouter, Gemini, HuggingFace) with automatic fallback
• Memory Layer: Redis (with file-based fallback) for incident history and similarity detection
• CLI Framework: Typer + Rich (professional terminal output)
• Cloud Integration: AWS (S3, CloudWatch), Kubernetes, HTTP endpoints
• AI Pattern: Multi-agent agentic architecture with 4 specialized agents
• Reasoning: Evidence-based decision making with P0/P1/P2/P3 severity classification
• Prompt Engineering: Robust JSON extraction with retry logic and safe parsing
• Testing: pytest with 45+ unit tests and integration test coverage

---

📋 Project Structure

opspilot/
├── agents/                    # Four specialized AI agents
│   ├── planner.py            # Hypothesis generation
│   ├── verifier.py           # Evidence-based verification
│   ├── fixer.py              # Safe fix suggestions
│   └── remediation.py        # 3-tier remediation plans
├── context/                   # Context gathering modules
│   ├── logs.py               # Log analysis
│   ├── env.py                # Environment variables
│   ├── deps.py               # Dependency detection
│   ├── docker.py             # Docker configuration
│   ├── project.py            # Project structure
│   ├── production_logs.py    # Multi-source log fetching (S3, K8s, CloudWatch)
│   ├── deployment_history.py # Git-based deployment correlation
│   └── pattern_analysis.py   # Error pattern detection & severity classification
├── utils/                     # Shared utilities
│   ├── llm_providers.py      # Multi-provider LLM router with fallback
│   └── llm.py                # Backward-compatible LLM wrapper
├── tools/                     # Evidence collection utilities
│   ├── log_tools.py          # Log error analysis
│   ├── env_tools.py          # Environment validation
│   └── dep_tools.py          # Dependency checking
├── diffs/                     # Domain-specific fix templates
├── memory.py                  # File-based memory (fallback)
├── memory_redis.py            # Redis-based memory with user isolation
├── tests/                     # Comprehensive test suite (45+ tests)
│   ├── test_pattern_analysis.py
│   ├── test_production_logs.py
│   ├── test_remediation.py
│   └── test_llm_providers.py
└── cli.py                     # Command-line interface

---

🔒 Safety & Design Principles

• Dry-Run Only: Never automatically applies changes to your code
• Evidence-Based: All suggestions backed by concrete evidence
• Confidence Scoring: Transparent about certainty levels (0.0 - 1.0)
• Privacy-Focused: Prefers local LLM (Ollama) with automatic fallback to cloud
• User Isolation: Redis memory uses SHA-256 project hashing for complete data separation
• Auto-Expiring Data: Incidents automatically expire after configurable TTL (default 30 days)
• High Availability: Multi-provider LLM system with automatic failover
• Modular Design: Easy to extend with new agents, providers, or context sources
• Production-Ready: Comprehensive test coverage, error handling, and graceful degradation

---

🛠️ Development

Running from Source

# Install in development mode
pip install -e .

# Run the CLI
opspilot analyze

Requirements

• Python 3.8+
• At least one LLM provider (see Quick Start)
• Optional: Redis for production incident memory

Running Tests

# Install test dependencies
pip install -e ".[dev]"

# Run all tests
pytest

# Run with coverage
pytest --cov=opspilot tests/

# Run specific test file
pytest tests/test_llm_providers.py

---

🗺️ Roadmap

• Multi-provider LLM support with automatic fallback
• Comprehensive test coverage (45+ tests)
• Redis-based memory with user isolation
• Production log fetching (S3, K8s, CloudWatch)
• Deployment correlation analysis
• Severity classification (P0/P1/P2/P3)
• Plugin system for custom agents
• Web API for remote usage
• More domain-specific fix templates (PostgreSQL, MongoDB, etc.)
• Real-time metrics dashboard
• Slack/PagerDuty integration for incident alerts

---

📄 License

MIT License - see LICENSE for details

---

🤝 Contributing

Contributions are welcome! This project is under active development. Please:

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Submit a pull request

For major changes, please open an issue first to discuss your ideas.

---

📧 Contact

For questions or feedback, please open an issue on GitHub.

---

Built with ❤️ using agentic AI principles

---

📦 PyPI

This package is available on PyPI: opspilot-ai