vibecheck 🔮

A little terminal tool that checks how much of a GitHub repo was actually written by AI.

It digs through commit messages, co-author tags, and project setup files to tell you:

1. How much AI code is in here? (0% to 100%)
2. What kind of project is this? (e.g. AI-built with human tests, a weekend prototype, or 100% hand-crafted)
3. Is it tested and reviewed, or just "vibe-coded"?

Quickstart

You don't need to install anything! Run directly with uvx:

uvx ai-vibecheck <github-repo>

Or install via pip:

pip install ai-vibecheck
vibecheck <github-repo>

Examples:

# Check any repo (e.g. open-metadata has ~60% AI co-authored commits)
vibecheck open-metadata/OpenMetadata

# Check an open-source project
vibecheck pallets/flask

# Or paste the full URL
vibecheck https://github.com/torvalds/linux

🌟 Cool Features

1. ⚔️ Side-by-Side Comparison

Compare two repositories head-to-head in a split-screen table:

vibecheck compare open-metadata/OpenMetadata pallets/flask

2. 🔀 Pull Request Vibe Check

Audit a specific Pull Request to see if it was written by an AI coding agent:

vibecheck pallets/flask --pr 5000

3. 🏆 Organization Leaderboard

Rank an entire GitHub organization's active repos by AI adoption:

# Rank top repos in an organization
vibecheck org pallets --limit 5
vibecheck org encode --limit 5

Plain-English Categories

• 🤖 AI-Built, Human-Tested: Lots of AI code (>75%), but backed by real tests, CI/CD, and code reviews.
• 🚀 Pure Vibe-Code: Built fast by prompting an AI in 1 or 2 big commits. No tests, just vibes.
• 🍝 AI Spaghetti: Lots of erratic commits like "fix bug", "try again", showing someone pasted terminal errors into an LLM until it compiled.
• 🛠️ 100% Hand-Crafted: Classic, hand-written software with human commit discipline and zero AI footprints.
• 🤝 Human + AI Tag Team: A healthy human codebase where engineers use AI assistants for parts of the work.

How It Works Under the Hood

1. GitHub login: Automatically uses your existing GitHub CLI session (gh auth). Higher rate limits and access to private repos you can view. Falls back to public HTTP if not logged in.
2. AI analysis: Runs via your local Gemini CLI session by default (no API keys required).
3. Forensic detection:
   • Finds agent steering files (CLAUDE.md, .cursorrules, .agents).
   • Parses git trailers (Generated with Claude Code, gemini-code-assist[bot], copilot[bot]).
   • Automatically filters out automated release bumps so metrics reflect real feature engineering.

Options

# Switch to Claude
vibecheck <repo> -p claude

# Switch to OpenAI
vibecheck <repo> -p openai -m gpt-4o

# Run tests
uv run pytest

# Check code quality
uv run ruff check .