diffsniff-gatekeeper 0.1.0

A local git gatekeeper that blocks AI-generated slop in PR descriptions.

DiffSniff

Stop AI-generated filler from sneaking into your Git history.

DiffSniff is a local, terminal-based gatekeeper that cross-references your staged Git changes against your Pull Request description. If it detects that the description is generic AI-generated filler that doesn't accurately reflect the code you've written, it blocks the commit.

---

Why DiffSniff?

Traditional AI detectors focus on writing style, which makes them easy to bypass. You can simply ask a model to "write like a human."

DiffSniff takes a different approach.

Instead of analyzing how text is written, it verifies whether the claims in your PR description actually match the code changes in your diff.

How It Works

Local Heuristics

DiffSniff performs lightweight local analysis to identify low-effort copy-paste descriptions by measuring:

• Vocabulary overlap
• Structural variance
• Content specificity

This catches obvious mismatches instantly without requiring an API call.

Adversarial Q&A

DiffSniff then:

1. Feeds your Git diff to an LLM.
2. Generates highly specific technical questions about the code changes.
3. Checks whether your PR description answers those questions.

If your PR claims one thing while the code does another, DiffSniff flags it.

---

Installation

Install directly from PyPI:

pip install diffsniff-gatekeeper

---

Configuration

Bring Your Own Model

DiffSniff uses LiteLLM under the hood, allowing you to use virtually any supported provider, including:

• Gemini
• OpenAI
• Anthropic
• Local models

Example: Gemini

Export your API key:

export GEMINI_API_KEY="your-google-ai-key"

Optional Configuration

Create a config.json file in your working directory to customize behavior:

{
  "model": "gemini/gemma-4-31b-it",
  "slop_threshold": 55,
  "num_questions": 3
}

Configuration Options

Option	Description
model	LLM used for adversarial questioning
slop_threshold	Minimum score required to pass validation
num_questions	Number of generated code-specific questions

Using OpenAI

To switch providers:

{
  "model": "gpt-4o"
}

Then export your API key:

export OPENAI_API_KEY="your-openai-key"

---

Usage

1. Stage your code changes:

git add .

2. Write a draft PR description in a Markdown file.

3. Run DiffSniff:

diffsniff pr_draft.md

---

Results

✅ Pass

Your PR description accurately reflects the code changes and you're ready to push.

❌ Fail

Your description doesn't sufficiently explain what the code actually does.

Rewrite the documentation and try again.

---

Philosophy

DiffSniff doesn't care whether a human or an AI wrote your PR description.

It cares whether the description is true.

If your documentation accurately explains the code, it passes.

If it's generic filler disconnected from reality, it fails.