checkod-ai 0.1.0

AI Impact Assessment for Code Changes

CheckodAI

AI Impact Assessment for code changes - understand what Copilot changed before you commit it.

Overview

CheckodAI is a Python CLI tool that helps developers understand the scope and implications of their code changes by analyzing git diffs and extracting modified symbols (functions, classes, variables). It can optionally use a local LLM (Ollama) to assess the risk of each change.

Privacy

CheckodAI runs entirely locally.

• No code is uploaded
• No cloud calls required
• No telemetry
• Works offline with local LLM (Ollama)

All analysis happens on your machine.

Key Features:

• 🚀 Local-first: Runs entirely on your machine - no cloud calls
• 🔍 Symbol-level analysis: Detects changed functions, classes, and variables
• 🏷️ Change type detection: Identifies what type of change (added, removed, modified, signature changed)
• 💡 Impact summaries: Human-readable recommendations for each changed symbol
• 🤖 AI-powered risk assessment: Uses local Ollama for intelligent impact analysis
• 🛡️ Commit guard: Advisory warnings for HIGH risk changes (non-blocking)
• 🎯 CLI-first interface: Simple checkod assess command
• 📊 Summary reports: Clear output of what changed and risk levels
• 🔧 Production-ready: Proper project structure and error handling
• ⚡ Graceful fallback: Works without Ollama using heuristic analysis

Installation

Prerequisites

• Python 3.8+
• Git
• Ollama (optional, for AI risk assessment)

Install Ollama (optional but recommended)

# macOS / Linux
curl -fsSL https://ollama.ai/install.sh | sh

# Or use Homebrew (macOS)
brew install ollama

# Pull the model
ollama pull llama3

# Start the server (runs on localhost:11434)
ollama serve

Quick Start

1. Install from PyPI:

pip install checkod-ai

2. Run assessment:

checkod assess

Or for development setup:

1. Clone the repository:

git clone <repository-url>
cd checkod

2. Create and activate a virtual environment:

python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install in development mode:

pip install -e .

Usage

Basic Usage

Analyze changes in the current repository:

checkod assess

Analyze changes in a specific repository:

checkod assess --repo /path/to/repo

Skip AI risk assessment (if Ollama not available):

checkod assess --no-risk

Commit Guard

The commit guard provides advisory warnings for HIGH risk changes:

# Assessment with advisory warnings enabled
checkod assess --risk

If HIGH risk is detected:

⚠️  HIGH IMPACT CHANGE DETECTED

This change may affect critical system behavior.

Recommended:
  • run related tests
  • verify downstream services
  • review impact summary

Commit will proceed.

Exit code: 0 (advisory, non-blocking)

See COMMIT_GUARD.md for detailed guard usage and integration patterns.

Example Output

🔍 Starting Impact Assessment...

📋 Git Diff (1250 characters):
────────────────────────────────────────────────────────────────────────────────
diff --git a/src/utils.py b/src/utils.py
index abc1234..def5678 100644
--- a/src/utils.py
+++ b/src/utils.py
@@ -10,6 +10,10 @@
+def calculateDiscount(price, tier):
+    return price * (1 - tier * 0.1)
...
────────────────────────────────────────────────────────────────────────────────

📊 Changed Symbols (3 detected):
  • calculateDiscount
  • OrderStatus
  • userTier

📈 Change Summary:
  Functions: 1 function added
  Classes: 1 class added
  Variables: 1 variable added

================================================================================
📋 Impact Summary
================================================================================
You changed: calculateDiscount()
Change Type: function added
Risk Level: MEDIUM

This may affect:
  • checkout_service (logic flow)
  • /api/orders (user-facing behavior)
  • test_checkout (test coverage)

Recommended Actions:
  • Write unit tests for calculateDiscount
  • Test integration with checkout service
  • Document function parameters and return type
  • Verify checkout_service handles the change
  • Load test affected service endpoints

✅ Assessment complete!

With AI risk assessment enabled (Ollama running), you'll also see:

================================================================================
🤖 AI Risk Assessment (powered by local Ollama)
================================================================================
📍 Analyzing risk for each symbol...

Symbol: calculateDiscount
────────────────────────────────
Risk Classification: MEDIUM

Reason: Used in multiple modules; requires testing and code review.

Validation Steps:
• Test discount calculation across pricing tiers
• Verify integration with checkout service
• Add regression tests if coverage is low

See Risk Assessment Module for setup and usage details.

Project Structure

checkod/
├── __init__.py           # Package initialization
├── main.py              # CLI entry point (Typer)
├── agent.py             # Core orchestrator
├── observe.py           # Git diff reading (GitPython)
├── understand.py        # Symbol extraction (regex patterns)
├── change_type.py       # Change type detection
├── assess.py            # Risk assessment (Ollama integration)
├── summary.py           # Impact summary generation
└── guard.py             # Commit guard for HIGH risk ⭐

Architecture

Components

1. main.py - CLI interface using Typer

   • Exposes assess command
   • Handles user input and options
   • Calls the agent orchestrator

2. agent.py - Orchestrator

   • Coordinates the workflow
   • Calls observe and understand modules
   • Formats and displays results

3. observe.py - Git integration

   • Uses GitPython to read diffs
   • Gets both staged and unstaged changes
   • Returns raw diff content

4. understand.py - Symbol extraction

   • Parses diff for changed symbols
   • Uses regex patterns for symbol detection
   • Returns list of modified symbols

5. change_type.py - Change type detection

   • Identifies what type of change was made
   • Classifies as: added, removed, modified, signature changed
   • Provides confidence scoring
   • Enriches symbols with metadata

6. assess.py - Risk assessment

   • Integrates with Ollama for AI analysis
   • Builds structured prompts for LLM
   • Provides heuristic fallback when Ollama unavailable
   • Classifies changes as LOW/MEDIUM/HIGH risk

Workflow

User Input → CLI (main.py)
    ↓
Agent (agent.py) - Orchestrator
    ↓
Observe (observe.py) - Read git diff
    ↓
Understand (understand.py) - Extract symbols
    ↓
Detect (change_type.py) - Identify change types
    ↓
Assess (assess.py) - Evaluate risk (optional, requires Ollama)
    ↓
Display Results

Development

Running Tests

pytest

Code Quality

# Format code
black .

# Check code style
flake8 .

# Type checking
mypy checkod

Development Dependencies

The project includes development tools for testing and code quality:

• pytest - Testing framework
• pytest-cov - Coverage reporting
• black - Code formatter
• flake8 - Linter
• mypy - Type checker

Install with:

pip install -e ".[dev]"

Current Limitations

Current Version (0.1.0)

• Regex-based parsing: Symbol extraction uses regex patterns, not full AST parsing
• Limited language support: Patterns work best with Python and JavaScript
• Basic LLM integration: Ollama support available but must be local
• No usage tracking: Doesn't analyze actual symbol usage in codebase

Future Enhancements

• Full AST-based parsing for accurate symbol detection
• Multi-language support (Java, Go, Rust, etc.)
• Dependency graph analysis for usage tracking
• Integration with codebase analysis tools
• Pre-commit hook integration
• Historical trend analysis
• Custom risk policies per team

Configuration

Future versions will support configuration via checkod.yaml:

# Planned for future release
symbols:
  min_impact_score: 0.5
languages:
  - python
  - javascript
patterns:
  exclude:
    - test_*
    - __pycache__

Troubleshooting

"Not a valid git repository" error

Ensure you're running checkod assess in a directory that is a git repository:

git init  # If needed

No symbols detected

This can happen if:

• Changes are only in configuration files
• Symbols don't match current regex patterns
• Consider checking the raw diff: git diff

Virtual environment issues

Recreate the environment:

rm -rf venv
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Documentation

• README.md - Full user guide, architecture, troubleshooting
• SETUP.md - Quick start and common tasks
• CONTRIBUTING.md - How to contribute and develop
• CHANGE_TYPE_DETECTION.md - Change type identification and usage
• RISK_ASSESSMENT.md - AI-powered risk assessment setup and usage
• AI_RISK_ASSESSMENT_SUMMARY.md - Implementation overview
• IMPACT_SUMMARY.md - Developer-facing impact analysis and recommendations
• COMMIT_GUARD.md - Commit blocking on HIGH risk changes
• .github/copilot-instructions.md - Copilot development guidance

Contributing

This is an experimental project. Contributions are welcome!

License

MIT License - See LICENSE file for details

Roadmap

Phase 1 (Current):

• Basic git diff reading
• Regex-based symbol extraction
• CLI interface

Phase 2 (Planned):

• Full AST parsing
• Multi-language support
• Impact scoring

Phase 3 (Future):

• AI-powered analysis
• Integration with CI/CD
• Web-based dashboard

---

Note: This is an early-stage project. The API and behavior are subject to change.