ai-logguard 1.0.1

🤖 AI-powered CLI for CI/CD log analysis with ML + LLM hybrid intelligence

🤖 AI-LogGuard

AI-powered Cross-platform CLI for CI/CD Log Analysis, Failure Prediction, and Auto-fix Suggestion

AI-LogGuard is an intelligent command-line tool that analyzes CI/CD build logs using ChatGPT and Machine Learning models to classify errors and provide fix suggestions.

🌟 Features

✅ Core Features

• 🤖 ChatGPT Integration: AI-powered log analysis and fix suggestions
• 🎯 Intelligent Error Classification: 9 error categories with ML models
• 📊 Model Comparison: RandomForest, XGBoost, LSTM, GRU, DistilBERT
• 🔧 Auto Fix Suggestions: Specific code fixes from ChatGPT
• 🔍 Multi-platform Support: Jenkins, GitHub Actions, GitLab CI

📊 Error Categories

Category	Description
dependency_error	Package/module resolution issues
syntax_error	Code syntax errors
test_failure	Unit/integration test failures
timeout	Operation exceeded time limits
permission_error	Access denied, auth failures
network_error	Connection issues, DNS failures
environment_error	Missing env vars, PATH issues
build_error	Compilation/linking errors
resource_error	Memory, disk space issues

🚀 Quick Start

Installation

# Clone repository
git clone https://github.com/SyLe-Van/AI-LogGuard.git
cd ai-logguard

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# Install
pip install -e .

# Set up OpenAI API key
export OPENAI_API_KEY='your-api-key'

CLI Usage

# View all commands
ailog --help

# Analyze a log file with ChatGPT
ailog ask build.log

# Get fix suggestions
ailog fix failed_build.log

# View model comparison
ailog models

# Run interactive demo
ailog demo

Example Commands

1. Analyze Log File

ailog analyze build_log.txt --llm

Example Output:

🔍 Analyzing log file: build_log.txt

======================================================================
🤖 AI ANALYSIS RESULT
======================================================================
**Error Type**: dependency_error

**Summary**: NPM failed to resolve peer dependency conflict between
React 18.2.0 and react-scripts 4.0.3 which requires React 17.

**Root Cause**: Version mismatch in package dependencies

**Fix Suggestions**:
1. Update react-scripts: npm install react-scripts@5
2. Or use --legacy-peer-deps flag
3. Check package.json for conflicts

**Severity**: Medium
======================================================================

2. Get Fix Suggestions

ailog fix error.log

3. View Model Performance

ailog models

Output:

📊 AI-LogGuard Model Comparison

┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Model        ┃ Type          ┃ Accuracy ┃ F1-Score ┃ Train Time ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━┩
│ RandomForest │ Classical ML  │   0.8234 │   0.8187 │      45.2s │
│ XGBoost      │ Classical ML  │   0.8456 │   0.8412 │      67.8s │
│ LSTM         │ Deep Learning │   0.8678 │   0.8634 │     234.5s │
│ GRU          │ Deep Learning │   0.8589 │   0.8545 │     198.3s │
│ DistilBERT   │ Transformer   │   0.9123 │   0.9095 │    1245.6s │
└──────────────┴───────────────┴──────────┴──────────┴────────────┘

🏆 Best Model: DistilBERT (F1: 0.9095)

4. Fetch from Jenkins

ailog fetch -p jenkins -u http://jenkins:8080 -j my-job -t $JENKINS_TOKEN

🏗 Architecture

Classification System

Log Input
    ↓
Pattern Matching Engine
    ├─ 60+ keyword patterns per category
    ├─ Context detection (npm, jest, docker, etc.)
    └─ Score calculation with weights
    ↓
Confidence Analysis
    ├─ Score distribution
    └─ Gap between top 2 categories
    ↓
Output: (category, confidence)

Why Rule-Based vs ML?

We tested both approaches extensively:

Approach	Test Accuracy	Real-world Accuracy	Cost	Speed
ML (85K dataset)	99.90%	0% ❌	$0	~50ms
Rule-based	N/A	100% ✅	$0	<5ms

Key Insight: ML models trained on synthetic logs suffer from severe overfitting. Rule-based patterns generalize better for CI/CD error classification.

Project Structure

ai-logguard/
├── src/
│   ├── main.py              # CLI entry point
│   ├── parse.py             # Basic log parsing
│   ├── ml/
│   │   └── classifier.py    # Rule-based classifier
│   └── fetchers/
│       ├── jenkins_fetcher.py
│       └── gitlab_fetcher.py
├── scripts/
│   ├── test_rule_based.py   # Classifier validation
│   ├── collect_github_automated.py  # Real log collection
│   └── train_*.py           # ML experiments (archived)
├── data/
│   ├── final_dataset/       # 85K logs (research)
│   └── real_logs_massive/   # 1K+ real GitHub Actions logs
├── tests/
│   └── sample_logs.txt
└── requirements.txt

🔬 Research & Development

Dataset Collection (Research Phase)

We collected and tested with:

• 1,081 real GitHub Actions logs (15 language/topic combinations)
• 16,008 LogHub system logs (12 datasets: HDFS, Spark, etc.)
• 82,904 synthetic logs (generated from real templates)
• Total: 85,509 samples, 1.0GB

ML Experiments Summary

Stage	Dataset	Test Acc	Real Acc	Finding
Baseline	300 synthetic	97.78%	N/A	Too small
Scale-up	10K synthetic	96.80%	40%	Overfitting
Massive	100K synthetic	99.78%	60%	Better but not enough
Final	85K (1K real + 16K LogHub + 83K synth)	99.90%	0%	Complete failure

Root Cause: 97% synthetic data dominated training. Model memorized patterns instead of learning generalizable features.

Pivot Decision: Switched to rule-based approach → 100% real-world accuracy

🛠 Development

Run tests

pytest

Run with coverage

pytest --cov=src --cov-report=html

Code formatting

black src/ tests/
isort src/ tests/

Type checking

mypy src/

📊 Example Output

🔍 Analyzing log file: tests/sample_logs.txt

✅ Log file loaded
✅ Log parsed successfully

╭─ 🔧 JENKINS - test-job ───────────────────────────────────────────╮
│ Build: N/A                                                         │
│ Status: ⚠️ UNSTABLE                                               │
│ Triggered by: Sy Le                                                │
╰────────────────────────────────────────────────────────────────────╯

                           📊 Statistics
┏━━━━━━━━━━━━━━━┳━━━━━━━┓
┃ Metric        ┃ Count ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━┩
│ Total Lines   │   121 │
│ Stages/Steps  │     1 │
│ Errors        │     5 │
│ Warnings      │     7 │
│ Retries       │     4 │
└───────────────┴───────┘

🎯 Stages/Steps
└── ✅ Simulate Build Logs - SUCCESS (5 errors, 7 warnings, 4 retries)

❌ Errors (showing 5 of 5)
┏━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Line ┃ Level   ┃ Message                                         ┃
┡━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 23   │ ERROR   │ [ERROR] Failed to fetch dependency: lib-xyz     │
│ 39   │ ERROR   │ [ERROR] Compilation failed in module: invoice   │
│ 53   │ ERROR   │ [ERROR] Unit test failed: test_order_checkout   │
│ 63   │ ERROR   │ [ERROR] DB connection timeout on first try      │
│ 85   │ ERROR   │ [ERROR] Deployment failed due to timeout        │
└──────┴─────────┴─────────────────────────────────────────────────┘

🎯 Roadmap

• Phase 1: Foundation & Basic Parsing (Week 1-2) ✅
  • CLI structure with Typer
  • Jenkins & GitHub Actions parsers
  • Unified log schema
  • Rich terminal output
• Phase 2: LLM Integration (Week 3-4)
  • OpenAI API integration
  • Prompt engineering
  • Summarization & explanation
  • Fix suggestions
• Phase 3: ML Model Training (Week 5-6)
  • Dataset collection
  • Feature engineering
  • Model training (Random Forest, Logistic Regression)
  • Hybrid ML + LLM pipeline
• Phase 4: Feedback Loop (Week 7-8)
  • User feedback collection
  • Analytics dashboard
  • Automated retraining
• Phase 5: Production Ready (Week 9-10)
  • Performance optimization
  • Packaging (PyPI, Docker)
  • Complete documentation

🤝 Contributing

Contributions are welcome! This is a thesis project, but suggestions and feedback are appreciated.

📄 License

MIT License - see LICENSE file for details

👤 Author

Sy Le Van

• GitHub: @SyLe-Van
• Thesis: AI-powered Cross-platform CLI for CI/CD Log Analysis

---

Note: This is an active thesis project. Features are being developed according to the roadmap.