pr-health-agent 2.0.3

Comprehensive GitHub PR health scoring with AI-powered analysis

PR Health Score Agent

A sophisticated CLI-based agent that calculates health scores for GitHub Pull Requests using multiple factors and AI-powered analysis.

🌟 Features

• Enhanced Interactive Menu System (v2.0.1)

  • ⌨️ Number-based selection: Press 1, 2, 3 instead of typing
  • 🎯 Multi-select support: Configure multiple AI providers at once (e.g., 1,2,3)
  • ✅ Smart defaults: Default values shown in brackets, just press Enter
  • 🔄 Consistent UX: All menus and prompts follow the same pattern
  • See MENU_GUIDE.md for detailed usage

• Comprehensive 13-Category Scoring System (100 Points)

  • Part A: PR Review Criteria (70 points)

    • 📏 Size (5): Multi-factor analysis (lines, files, commits, distribution)
    • 🧪 Test Quality (10): Coverage % + delta analysis
    • 📝 Commit Quality (5): Conventional commits + best practices
    • 👀 Review Quality (5): Reviewers, approvals, discussion engagement
    • 🔒 Security (10): Secrets detection + vulnerability scanning
    • 🎯 Code Quality (8): Complexity, duplication, function length
    • 💅 Code Style (3): Linting errors, formatting adherence
    • 📦 Dependencies (4): Vulnerability detection via Dependabot API
    • 📋 CHANGELOG (5): Updates, versioning, breaking changes
    • 👥 CODEOWNERS (3): File coverage, ownership clarity
    • ⚠️ Breaking Changes (5): Detection + documentation quality
    • 📝 Documentation (7): PR description, README, inline docs

  • Part B: Impact Analysis (30 points) - AI-Powered

    • 🔍 Complexity (10): Architectural complexity, cognitive load
    • 🎯 Impact Scope (10): Business impact + technical reach
    • ⚡ Performance (10): Execution time, resource usage, database impact

• AI-Powered Deep Analysis (Multiple Providers)

  • 🤖 Google Gemini 2.0 Flash (Recommended - Free tier, fast)
  • ☁️ AWS Bedrock Claude 3.5 Haiku (Supports SSO, API Key, IAM Role)
  • 🔮 OpenAI GPT-4o-mini (Cost-effective)
  • Analyzes actual code changes (not just filenames)
  • Complexity assessment with cognitive load analysis
  • Business and technical impact evaluation
  • Performance implications (database, API, resource usage)
  • Automated quality recommendations with score improvement values

• Dependency Security Scanning

  • 🔍 Uses GitHub Dependabot API (FREE - no GHAS license required)
  • Detects vulnerabilities: Critical, High, Moderate, Low
  • Supports 9+ ecosystems: npm, pip, maven, gradle, composer, bundler, cargo, go, nuget
  • Auto-scoring: Critical = 0pts, High = -20pts, Moderate = -8pts, Low = -3pts

• Automated Actions

  • 💡 Recommendations Engine: Shows actionable improvements with point values
  • 🚫 Merge Blocking: Prevents merges below score threshold (70/100)
  • 🏷️ Label Management: Auto-labels PRs based on health category
  • 💬 PR Comments: Posts detailed health reports with all 13 categories
  • ✅ Status Checks: Creates commit status checks

📋 Prerequisites

• Python 3.8+
• GitHub Personal Access Token with repo and security_events scope
• AI Provider (Optional, choose one or more):
  • Google Gemini API Key (Recommended - free tier, fast)
  • AWS Bedrock (API Key, SSO, or IAM Role)
  • OpenAI API Key

📊 Comprehensive Scoring System

The PR Review Agent uses a 100-point comprehensive scoring system with 13 evaluation categories:

Part A: PR Review Criteria (70 points)

Category	Max Points	Evaluation Criteria	Data Source
📏 Size	5	Lines changed, files modified, commits count, distribution	Enhanced Size Analyzer
🧪 Test Quality	10	Coverage %, coverage delta, test file ratio	Coverage Analyzer
📝 Commit Quality	5	Conventional commits, message quality, issue references	Commit Analyzer
👀 Review Quality	5	Reviewers count, approvals, review comments	PR Metadata
🔒 Security	10	Secrets detection, vulnerability scanning, insecure patterns	Security Scanner + Dependabot
🎯 Code Quality	8	Complexity, duplication, long functions, maintainability	Code Quality Analyzer
💅 Code Style	3	Linting errors, formatting adherence	Style Checkers
📦 Dependencies	4	Vulnerabilities (Critical/High/Moderate/Low)	GitHub Dependabot API
📋 CHANGELOG	5	File exists, updated, versioning, breaking changes	CHANGELOG Analyzer
👥 CODEOWNERS	3	File coverage %, ownership clarity	CODEOWNERS Analyzer
⚠️ Breaking Changes	5	Detection, documentation, migration guide	Breaking Changes Analyzer
📝 Documentation	7	PR description, title, testing info, context	Documentation Analyzer

Part B: Impact Analysis (30 points) - AI-Powered

Category	Max Points	Evaluation Criteria	AI Provider
🔍 Complexity	10	Architectural complexity, cognitive load, design patterns	Bedrock/Gemini/OpenAI
🎯 Impact Scope	10	Business impact, technical reach, affected modules	Bedrock/Gemini/OpenAI
⚡ Performance	10	Execution time, resource usage, database queries, API calls	Bedrock/Gemini/OpenAI

Score Categories

Score Range	Category	Merge Decision
90-100	🌟 Excellent	✅ Merge Recommended
80-89	✅ Good	✅ Merge Recommended
70-79	⚠️ Needs Improvement	✅ Merge Allowed
60-69	🔴 Poor	🚫 Merge Blocked
0-59	🚫 Critical Issues	🚫 Merge Blocked

Dependency Vulnerability Scoring

Severity	Point Deduction	Auto-Fail
Critical	N/A	❌ Score = 0
High	-20 per vulnerability	No
Moderate	-8 per vulnerability	No
Low	-3 per vulnerability	No

📦 Installation

For End Users (No Code Access Required)

# Install directly from GitHub
pip install git+https://github.com/LTIM-Digital-Engineering/PR-Review-Agent.git

# With AWS Bedrock support
pip install "git+https://github.com/LTIM-Digital-Engineering/PR-Review-Agent.git#egg=pr-health-agent[bedrock]"

After installation, use these commands:

# Interactive application
pr-health-app

# Setup wizard
pr-health-interactive

# Direct analysis
pr-health owner/repo 123 --ai-analysis --post-comment

📖 See INSTALLATION.md for complete installation guide

For Developers (Local Development)

🚀 Quick Start

1. Navigate to Directory

cd C:\Users\10691618\Desktop\PR-Review-Agent

2. Install Dependencies

pip install -r requirements.txt

For AWS Bedrock support (optional):

pip install boto3

3. Run Unified Application

python app.py

This launches an interactive menu with options for:

1. Setup & Configuration - Configure GitHub PAT and AI providers
2. Analyze Pull Request - Run comprehensive PR analysis
3. Commands Reference - View all available commands
4. Exit

4. Alternative: Interactive Setup Wizard

python interactive.py

This wizard will guide you through:

• ✅ GitHub token setup and validation
• ✅ AI provider selection (Gemini/Bedrock/OpenAI)
• ✅ API key validation
• ✅ Environment variable configuration

5. Direct Command-Line Usage

# Basic analysis
python pr_agent.py owner/repo PR_NUMBER

# With AI analysis
python pr_agent.py owner/repo PR_NUMBER --ai-analysis

# Full analysis with comment posting
python pr_agent.py owner/repo PR_NUMBER --ai-analysis --post-comment

⚙️ Configuration

Scoring Weights (config.yaml)

Category	Weight	What It Measures
Size	10%	Lines, files, commits (multi-factor)
Commit Quality	10%	Message quality, conventional commits
Test Quality	20%	Coverage %, test files ratio
Review Quality	20%	Reviewers, approvals, comments
Complexity	15%	AI-powered complexity assessment
Security	15%	Secret scanning, vulnerabilities
Code Quality	5%	Linting, complexity, duplication
Documentation	5%	PR description quality

PR Size Categories (Multi-Factor)

Category	Lines	Files	Commits	Score
Small	<5	1-2	1-2	100 pts
Medium	<10	4-5	≤5	85 pts
Complex	15-50	<10	≤10	70 pts
Very Complex	>50	>10	>10	50 pts

AI Provider Configuration

Edit config.yaml to configure your AI provider:

ai:
  # Gemini (Google)
  gemini:
    model: "gemini-2.0-flash-lite"
    max_tokens: 2000
    temperature: 0.3
  
  # Bedrock (AWS)
  bedrock:
    model_id: "us.anthropic.claude-3-5-haiku-20241022-v1:0"
    region: "us-east-1"
    auth_method: "api_key"  # or 'sso', 'iam_role'
    sso_profile: ""  # For SSO authentication
  
  # OpenAI
  openai:
    model: "gpt-4o-mini"
    max_tokens: 2000
    temperature: 0.3

Environment Variables

Set these in your system or create a .env file:

# Required
GITHUB_TOKEN=ghp_your_github_token

# AI Providers (choose one or more)
GEMINI_API_KEY=your_gemini_api_key
OPENAI_API_KEY=your_openai_api_key

# AWS Bedrock (choose one auth method)
BEDROCK_API_KEY=your_bedrock_key
# OR
AWS_PROFILE=your_sso_profile_name
# OR use default IAM role

# Optional
AI_PROVIDER=gemini  # or bedrock, openai
AWS_REGION=us-east-1

📚 Documentation

• AI_PROVIDERS.md - Detailed AI provider setup guide
• config.yaml - Full configuration reference with comments

🔧 Project Structure

PR-Review-Agent/
├── app.py                    # Main unified application entry point
├── pr_agent.py              # CLI tool for direct PR analysis
├── interactive.py           # Setup wizard
│
├── pr_collector.py          # GitHub API integration
├── score_calculator.py      # Health score calculation engine
├── ai_analyzer.py           # Multi-provider AI integration
├── security_scanner.py      # Security vulnerability scanning
├── coverage_analyzer.py     # Code coverage analysis
├── commit_analyzer.py       # Commit quality analysis
├── code_quality_analyzer.py # Code quality metrics
├── comment_generator.py     # PR comment formatting
├── merge_controller.py      # Merge blocking logic
│
├── config.yaml              # Configuration file
├── requirements.txt         # Python dependencies
├── .env.example            # Environment template
│
└── AI_PROVIDERS.md         # AI provider setup guide

🛡️ Security & Dependency Scanning

Security Scanner Features

The security scanner detects:

• Secrets: GitHub tokens, AWS keys, API keys, private keys, SSH keys
• Credentials: Hardcoded passwords, JWT tokens, OAuth tokens, Stripe keys
• Vulnerabilities: SQL injection patterns, XSS risks, command injection
• Insecure Functions: eval(), exec(), pickle.loads(), md5(), sha1()
• Configuration Issues: Debug mode enabled, insecure SSL settings

Dependency Vulnerability Scanning

Uses GitHub Dependabot API (FREE - no GitHub Advanced Security license required):

• Supported Ecosystems: npm, pip, maven, gradle, composer, bundler, cargo, go, nuget
• Severity Detection: Critical, High, Moderate, Low
• Automated Scoring:
  • Critical vulnerabilities = Auto-fail (0 points)
  • High severity = -20 points each
  • Moderate severity = -8 points each
  • Low severity = -3 points each
• Requirements:
  • GitHub PAT with security_events scope
  • Dependabot enabled in repository settings

🧪 Testing

Validate your setup:

# Check dependencies and configuration
python setup_check.py

# Test Gemini connection
python test_gemini.py

# Test Bedrock connection
python test_bedrock.py

📊 Example Output

📊 COMPREHENSIVE ANALYSIS COMPLETE
======================================================================
Overall Score: 78.5/100

PR Review Criteria: 55.2/70
  - PR Structure: 18.5/25
  - Code Standards: 20.0/25
  - Documentation: 16.7/20

Impact Analysis: 23.3/30
  - Complexity: 8.0/10
  - Impact Scope: 7.5/10
  - Performance: 7.8/10

Category: GOOD
======================================================================

## 📊 Part A: PR Review Criteria (55.2/70)

### 1️⃣ PR Structure & Quality (18.5/25)

📏 **Size** (3.8/5)
Lines: 450 | Files: 12 | Commits: 5
Category: Good - Manageable PR size

🧪 **Test Quality** (7.5/10)
Coverage: 75% | Delta: +5% | Test Files: 4/12

📝 **Commit Quality** (4.2/5)
Conventional: 80% | Issue References: Yes

👀 **Review Quality** (3.0/5)
Reviewers: 2 | Approvals: 1 | Comments: 8

### 2️⃣ Code Quality & Standards (20.0/25)

🔒 **Security** (8.5/10)
No secrets detected | 2 moderate vulnerabilities

🎯 **Code Quality** (6.5/8)
Complex Functions: 3 | Long Functions: 2

💅 **Code Style** (2.0/3)
Linting: Minor issues

📦 **Dependencies** (3.0/4)
2 vulnerabilities (Moderate)

### 3️⃣ Documentation & Standards (16.7/20)

📋 **CHANGELOG** (4.0/5)
Updated with version and changes

👥 **CODEOWNERS** (2.5/3)
Coverage: 85%

⚠️ **Breaking Changes** (3.2/5)
1 breaking change documented

📝 **Documentation** (7.0/7)
Complete PR description

## 🎯 Part B: Impact Analysis (23.3/30)

🔍 **Complexity** (8.0/10) - AI Analysis
Level: Moderate
Assessment: Well-structured changes with clear separation of concerns

🎯 **Impact Scope** (7.5/10) - AI Analysis  
Level: Medium
Affected: Authentication module, 3 API endpoints

⚡ **Performance** (7.8/10) - AI Analysis
Impact: Neutral to Slight Improvement
Database queries optimized

## 💡 Recommendations - Score Improvement Opportunities

- **Testing**: Improve test coverage from 75% to at least 80%. **(+2.5 points)**
- **Dependencies**: Update 2 vulnerable dependencies to secure versions. **(+1.0 points)**
- **Security**: Fix moderate vulnerabilities in dependencies. **(+1.5 points)**
- **Code Quality**: Refactor 3 complex functions **(+1.5 points)**
- **CODEOWNERS**: Improve coverage from 85% to 100% **(+0.5 points)**

## ✅ Summary

No critical issues detected. Address recommendations for higher score.

---
✅ **Merge Recommended** - Score meets minimum threshold (70)

🤝 Contributing

Contributions welcome! Please ensure:

• Code follows existing style
• Tests pass
• Documentation is updated

📄 License

MIT License - See LICENSE file for details

🆘 Support

For issues or questions:

1. Check AI_PROVIDERS.md for AI setup
2. Run python setup_check.py to validate setup
3. Review config.yaml for configuration options

🎯 Roadmap

• Integration with more CI/CD platforms
• Support for GitLab and Bitbucket
• Custom rule engine for security scanning
• Machine learning-based recommendations
• Web dashboard for analytics
• Test coverage expectations
• Comment templates

💻 Usage

Interactive Mode

For a guided experience with prompts:

python interactive.py

This will interactively ask you for:

• Repository name
• PR number
• Whether to use AI analysis
• Whether to post comments
• Custom threshold (optional)

Basic Usage

Analyze a PR without AI features:

python pr_agent.py owner/repo 123

With AI Analysis

Enable AI-powered complexity and impact analysis:

python pr_agent.py owner/repo 123 --ai-analysis

Post Comment to PR

Analyze and post results as a PR comment:

python pr_agent.py owner/repo 123 --ai-analysis --post-comment

Full Featured Analysis

Run complete analysis with all features:

python pr_agent.py owner/repo 123 --ai-analysis --post-comment --add-labels

Custom Threshold

Override the minimum passing score:

python pr_agent.py owner/repo 123 --threshold 80

Interactive Mode

For a guided experience with prompts:

python interactive.py

This will interactively ask you for repository, PR number, and analysis options.

CI/CD Integration

Use in CI/CD pipelines (exits with code 1 if score fails):

python pr_agent.py owner/repo 123 --ai-analysis --post-comment --add-labels
if ($LASTEXITCODE -ne 0) {
    Write-Host "PR health check failed!"
    exit 1
}

🎯 Command Line Options

Option	Description
repository	Repository in format owner/repo (required)
pr_number	Pull request number (required)
--config FILE	Path to config file (default: config.yaml)
--threshold NUM	Override minimum passing score
--ai-analysis	Enable AI-powered analysis using Bedrock
--post-comment	Post health report as PR comment
--add-labels	Add health-check labels to PR
--add-status	Add commit status check
--no-block	Don't exit with error if score fails
--output FORMAT	Output format: console, json, or both
--verbose, -v	Enable verbose output

📊 Scoring System

Total Score: 100 points

Category	Weight	Description
Size	20	Smaller PRs score higher
Test Coverage	25	Ratio of test files to code files
Review Quality	25	Number of reviewers, approvals, comments
Complexity	20	AI-assessed code complexity
Description	10	PR title and description quality

Thresholds:

• ✅ Excellent: 90+ points
• ✅ Good: 70+ points (default passing)
• ⚠️ Needs Improvement: 50-69 points
• ❌ Poor: < 50 points

🔧 Configuration

The config.yaml file controls all aspects of scoring and behavior:

thresholds:
  min_passing_score: 70
  excellent_score: 90

weights:
  size: 20
  test_coverage: 25
  review_quality: 25
  complexity: 20
  description_quality: 10

size_limits:
  small: 100
  medium: 500
  large: 1000

test_coverage:
  min_test_files_ratio: 0.3
  test_file_patterns:
    - "*test*.py"
    - "*_test.py"
    - "test_*.py"

🤖 AI Features

When --ai-analysis is enabled, the agent uses Amazon Bedrock to:

1. Complexity Analysis

   • Evaluates code structure and dependencies
   • Identifies risk areas
   • Provides refactoring suggestions

2. Impact Assessment

   • Determines affected system areas
   • Flags breaking changes
   • Identifies database/API changes

3. Review Comments

   • Generates constructive feedback
   • Highlights best practices
   • Raises security concerns

📝 Example Output

╔══════════════════════════════════════════════════════════╗
║           PR HEALTH SCORE REPORT                         ║
╚══════════════════════════════════════════════════════════╝

Overall Score: 85/100 (GOOD)

Breakdown:
  📏 Size:           18/20
  🧪 Test Coverage:  22/25
  👀 Review Quality: 20/25
  🔍 Complexity:     17/20
  📝 Description:    8/10

Status: ✅ PASS
Threshold: 70

🔐 GitHub Token Permissions

Your GitHub token needs these permissions:

• repo (full repository access)
• repo:status (for commit status checks)

🚀 CI/CD Integration Examples

GitHub Actions

name: PR Health Check

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  health-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      
      - name: Install dependencies
        run: pip install -r requirements.txt
      
      - name: Run PR Health Check
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          AWS_REGION: ${{ secrets.AWS_REGION }}
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        run: |
          python pr_agent.py ${{ github.repository }} ${{ github.event.pull_request.number }} \
            --ai-analysis --post-comment --add-labels

Azure DevOps

trigger:
  - none

pr:
  - main
  - develop

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: UsePythonVersion@0
  inputs:
    versionSpec: '3.11'

- script: pip install -r requirements.txt
  displayName: 'Install dependencies'

- script: |
    python pr_agent.py $(Build.Repository.Name) $(System.PullRequest.PullRequestNumber) \
      --ai-analysis --post-comment --add-labels
  displayName: 'Run PR Health Check'
  env:
    GITHUB_TOKEN: $(GITHUB_TOKEN)
    AWS_REGION: $(AWS_REGION)
    AWS_ACCESS_KEY_ID: $(AWS_ACCESS_KEY_ID)
    AWS_SECRET_ACCESS_KEY: $(AWS_SECRET_ACCESS_KEY)

🛠️ Troubleshooting

Common Issues

Issue: "Error: Missing required environment variables: GITHUB_TOKEN"

• Solution: Create a .env file with your GitHub token

Issue: "Failed to fetch PR data: 404"

• Solution: Check repository name format (owner/repo) and PR number

Issue: "Bedrock API error"

• Solution: Verify AWS credentials and Bedrock access in your region

Issue: "Failed to post comment: 403"

• Solution: Ensure GitHub token has repo permissions

Debug Mode

Run with verbose output:

python pr_agent.py owner/repo 123 --verbose

📦 Project Structure

PR-Review-Agent/
├── pr_agent.py              # Main CLI entry point
├── pr_collector.py          # GitHub API integration
├── score_calculator.py      # Health score calculation
├── bedrock_analyzer.py      # Amazon Bedrock AI integration
├── comment_generator.py     # PR comment formatting
├── merge_controller.py      # Merge blocking logic
├── config.yaml              # Configuration file
├── requirements.txt         # Python dependencies
├── .env.example            # Environment template
├── .gitignore              # Git ignore rules
└── README.md               # This file

🤝 Contributing

Contributions are welcome! Areas for improvement:

• Additional AI model support
• More scoring factors
• Enhanced test detection patterns
• Custom plugin system

📄 License

This project is provided as-is for educational and commercial use.

🙏 Acknowledgments

• GitHub API via PyGithub
• Amazon Bedrock for AI capabilities
• Claude 3.5 Sonnet for intelligent analysis

📞 Support

For issues or questions:

1. Check the troubleshooting section
2. Review configuration settings
3. Run with --verbose flag for detailed logs

---

Made with ❤️ for better Pull Requests