vibemetric 0.1.0

AI Coding Analytics Platform - Measure AI's impact on developer productivity

Vibemetric

AI Code Detection Platform - Measure AI's impact on developer productivity and track AI tool adoption across your engineering team.

Overview

Vibemetric is an enterprise-grade AI code detection platform that helps engineering managers:

• 📊 Track AI Adoption - Identify which developers are using AI tools and when they started
• 🚀 Measure Productivity Impact - Quantify velocity changes after AI tool adoption
• 🔍 Detect AI-Generated Code - Analyze code patterns with 4-layer detection system
• 📈 Generate Insights - Get actionable recommendations for team AI usage policies

Key Features

4-Layer Detection System

1. Artifact Detector (90% accuracy) - Detects AI tool configuration files

   • Supports 12 AI tools: Cursor, Claude, Kiro, GitHub Copilot, Aider, and more
   • Extracts adoption dates and author information from git history

2. Velocity Analyzer (80% accuracy) - Detects coding speed changes

   • Identifies 1.8x+ velocity spikes indicating AI adoption
   • Calculates baseline vs current velocity (lines/day)

3. Pattern Detector (70% accuracy) - Analyzes AI code patterns

   • AI-style comments and comprehensive docstrings
   • Conventional commits and structured PR descriptions
   • Type hints, dataclass usage, and code uniformity

4. ML Detector (85% accuracy) - Statistical pattern detection

   • Trained on DROID dataset (846k samples)
   • Detects GPT-4, Copilot, and other AI-generated code
   • Analyzes whitespace, entropy, and linguistic patterns

Enterprise-Ready CLI

# Scan any repository
vibemetric scan /path/to/repo

# JSON output for CI/CD integration
vibemetric scan . --format json

# Analyze more files for accuracy
vibemetric scan . --sample-size 50

Installation

Requirements

• Python 3.9 or higher
• Git (for repository analysis)

Install from Source

# Clone the repository
git clone https://github.com/salim8898/vibemetric.git
cd vibemetric

# Install with ML support
pip install -e ".[ml]"

# Or install for development
pip install -e ".[dev,ml]"

Verify Installation

vibemetric --help

Quick Start

Scan a Repository

# Scan current directory
vibemetric scan .

# Scan specific repository
vibemetric scan /path/to/repo

# Get JSON output
vibemetric scan . --format json

Example Output

╔═══════════════════════════════════════════════════════════╗
║              VIBEMETRIC SCAN RESULTS                      ║
╚═══════════════════════════════════════════════════════════╝

Repository: /Users/dev/my-project

Detection Summary
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
┃ Layer              ┃ Score    ┃ Status            ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
│ Artifact Detection │ 90.0/100 │ ✓ 2 tools         │
│ Velocity Detection │ 80.0/100 │ ✓ Spike           │
│ Pattern Detection  │ 75.0/100 │ ✓ 8 high-AI files │
│ ML Detection       │ 85.0/100 │ ✓ 12 AI files     │
└────────────────────┴──────────┴───────────────────┘

Combined AI Likelihood: 82.5/100
AI Assistance Level: SUBSTANTIAL
Confidence: 0.89

Interpretation:
This code shows SUBSTANTIAL AI assistance (82.5/100). Multiple 
detection layers confirm significant AI tool usage. Likely generated 
with tools like GPT-4, Copilot, or Claude.

Recommendations:
• Review AI tool usage policies with team
• Ensure AI-generated code is properly reviewed
• Consider code quality audits for AI-assisted code
• Document which AI tools are approved for use

Python Library Usage

from vibemetric.detectors import (
    ArtifactDetector,
    VelocityAnalyzer,
    PatternDetector,
    MLDetector
)
from vibemetric.scorer import Scorer

# Initialize detectors
artifact_detector = ArtifactDetector("/path/to/repo")
velocity_analyzer = VelocityAnalyzer("/path/to/repo")
pattern_detector = PatternDetector()
ml_detector = MLDetector()

# Run detection
artifacts = artifact_detector.detect()
artifact_signal = artifact_detector.get_detection_signal(artifacts)

velocity_metrics = velocity_analyzer.analyze()
velocity_signal = velocity_analyzer.get_detection_signal(velocity_metrics)

# Combine signals
scorer = Scorer()
vibe_score = scorer.calculate_vibe_score([
    artifact_signal,
    velocity_signal,
    # ... other signals
])

print(f"AI Likelihood: {vibe_score.overall_score:.1f}/100")
print(f"Assistance Level: {vibe_score.ai_assistance_level.value}")
print(f"Confidence: {vibe_score.confidence:.2f}")

AI Assistance Levels

Vibemetric classifies code into three assistance levels:

• MINIMAL (0-40): Primarily human-authored with little to no AI tool usage
• PARTIAL (40-70): Mixed human-AI collaboration, some AI assistance detected
• SUBSTANTIAL (70-100): Significant AI contribution, multiple AI indicators

Supported AI Tools

Vibemetric detects configuration files for:

• Cursor (.cursorrules)
• Claude (.claude/, claude.md, .anthropic/)
• Kiro (.kiro/)
• GitHub Copilot (.copilot/, .github/copilot/)
• Aider (.aider/)
• Windsurf (.windsurf/)
• Tabnine (.tabnine/)
• Codeium (.codeium/)
• Amazon CodeWhisperer (.aws/codewhisperer/)
• Replit Ghostwriter (.replit)
• Sourcegraph Cody (.cody/)
• JetBrains AI (.idea/ai/)

CLI Options

vibemetric scan [PATH] [OPTIONS]

Options:
  --format [terminal|json]  Output format (default: terminal)
  --sample-size INTEGER     Number of files to analyze (default: 10)
  --verbose                 Show detailed progress
  --help                    Show help message

Development

Setup Development Environment

# Install with development dependencies
pip install -e ".[dev,ml]"

# Run tests
pytest

# Run tests with coverage
pytest --cov=vibemetric --cov-report=html

# Run specific test suite
pytest tests/test_scorer.py -v

Project Structure

vibemetric/
├── src/vibemetric/
│   ├── cli.py                 # CLI interface
│   ├── models.py              # Data models
│   ├── scorer.py              # Score combination engine
│   ├── detectors/
│   │   ├── artifact_detector.py
│   │   ├── velocity_analyzer.py
│   │   ├── pattern_detector.py
│   │   └── ml_detector.py
│   └── ml/
│       └── feature_extractor.py
├── tests/                     # Test suite
├── models/                    # Trained ML models
└── pyproject.toml            # Project configuration

How It Works

Detection Process

1. Artifact Detection - Scans for AI tool config files in repository
2. Velocity Analysis - Analyzes commit history for velocity spikes
3. Pattern Detection - Examines code for AI-specific patterns
4. ML Detection - Uses trained model for statistical analysis
5. Score Combination - Weighted averaging with confidence calculation

Scoring Algorithm

Combined Score = (Artifact × 0.40) + (Velocity × 0.25) + 
                 (Pattern × 0.20) + (ML × 0.15)

Weights are based on layer accuracy and reliability:

• Artifact: 40% (most reliable - actual config files)
• Velocity: 25% (good for adoption timing)
• Pattern: 20% (catches specific AI patterns)
• ML: 15% (fallback for subtle patterns)

Known Limitations

• Squash Merge Workflows: Velocity analysis may be inaccurate for repos using squash merges. PR-based analysis coming in Phase 2.
• ML Model: Trained on DROID dataset, may have false positives on very clean human code
• Sample Size: Default 10-file sample may miss patterns in large repos. Use --sample-size 50 for better accuracy.

See KNOWN_ISSUES.md for details.

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

License

MIT License - See LICENSE file for details.

Support

• 🐛 Issues: GitHub Issues
• 💬 Discussions: GitHub Discussions

Citation

If you use Vibemetric in your research, please cite:

@software{vibemetric2026,
  title = {Vibemetric: AI Code Detection Platform},
  author = {Salim Shaikh},
  year = {2026},
  url = {https://github.com/salim8898/vibemetric}
}

Acknowledgments

Built with:

• GitPython - Git repository analysis
• scikit-learn - Machine learning
• Rich - Terminal formatting
• DROID Dataset - ML training data

---

Made with ❤️ by the Vibemetric team