Skip to main content

Detect duplicate PRs in GitHub repos

Project description

bepo

Detect duplicate pull requests in GitHub repos.

No ML, no embeddings, no API keys. Just static analysis of diffs.

A maintainer with 100 open PRs can run bepo check --repo foo/bar and in 5 minutes get a ranked list of "you should look at these pairs." That saves hours of manual review.

The Problem

Large repos waste engineering time on duplicate PRs. When multiple contributors fix the same bug independently, only one PR gets merged — the rest is wasted effort.

This actually happens. We analyzed 100 PRs from OpenClaw and found:

Cluster PRs What happened
Matrix startup bug 4 PRs 4 engineers independently fixed startupGraceMs = 05000
Media token regex 2 PRs Identical fix submitted twice
Feishu bitable config 2 PRs Same multi-account config fix

8 duplicate PRs across 3 bug fixes. That's real engineering time wasted.

Proof: OpenClaw Analysis

We ran bepo on OpenClaw's open PRs. Here's what it found:

$ bepo check --repo openclaw/openclaw --limit 100

#20025 <-> #19973
  Similarity: 86%
  Reason: Both fix #19843      ← Same issue!

#19868 <-> #19855
  Similarity: 81%
  Reason: Same files: parse.ts, pi-embedded-subscribe.tools.ts

#19871 <-> #19853
  Similarity: 100%
  Reason: Same files: bitable.ts, config-schema.ts, tools-config.ts

Verified manually:

PR Pair Similarity Verdict
#20025 ↔ #19973 (Matrix) 86% ✅ TRUE DUPLICATE — both change startupGraceMs from 0 to 5000
#19868 ↔ #19855 (regex) 81% ✅ TRUE DUPLICATE — identical PR titles
#19871 ↔ #19853 (Feishu) 100% ✅ TRUE DUPLICATE — same files, same fix
#19996 ↔ #19993 (unrelated) 20% ✅ Correctly NOT flagged

Precision: 80% (4/5 flagged clusters were true duplicates)

More Examples

VSCode — Found PRs touching same files for same feature:

#295823 <-> #295822
  Similarity: 77%
  Reason: Same files: chatModel.ts, chatForkActions.ts

  Both: "Use metadata flag for fork detection"

Next.js — Found related test updates:

#90121 <-> #90120
  Similarity: 86%
  Reason: Same files: test/

Install

pip install bepo

Requires GitHub CLI (gh) to be installed and authenticated.

Usage

# Check a repo for duplicate PRs
bepo check --repo owner/repo

# Adjust sensitivity (default: 0.4, higher = stricter)
bepo check --repo owner/repo --threshold 0.5

# Check more PRs
bepo check --repo owner/repo --limit 100

# JSON output for CI
bepo check --repo owner/repo --json

How It Works

bepo fingerprints each PR by extracting:

Signal Weight What it catches
Same issue ref (#123) 10.0 Definite duplicate
Same code changes 8.0 Identical lines added/removed
Same files touched 6.0 PRs modifying same code
Same feature domain 3.0 auth, messaging, database, etc.
Same imports 1.0 Similar dependencies

Then computes pairwise Jaccard similarity.

That's it. No embeddings, no LLM calls. Just:

  • Parse +++ b/path from diffs
  • Regex for #\d+ issue refs
  • Compare actual code changes
  • Set intersection for similarity

~300 lines of Python.

As a Library

from bepo import fingerprint_pr, find_duplicates

# Fingerprint PRs
fp1 = fingerprint_pr("#123", diff1, title="Fix auth", body="Fixes #456")
fp2 = fingerprint_pr("#124", diff2, title="Auth fix", body="Fixes #456")

# Find duplicates
dups = find_duplicates([fp1, fp2], threshold=0.4)
for d in dups:
    print(f"{d.pr_a}{d.pr_b}: {d.similarity:.0%}")
    print(f"  Shared issues: {d.shared_issues}")
    print(f"  Shared files: {d.shared_files}")

GitHub Action

name: PR Duplicate Check
on: [pull_request]

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - run: pip install bepo
      - run: bepo check --repo ${{ github.repository }} --json
        env:
          GH_TOKEN: ${{ github.token }}

Why This Works

Duplicates share obvious signals:

  • Same code = Identical changes (639 shared lines caught SoundChain duplicates)
  • Same issue ref = Same bug report (#19843 appeared in 4 Matrix PRs)
  • Same files = Same bug location (100% overlap for Feishu cluster)

Code overlap and issue refs catch most duplicates. Simple works.

Origin Story

This tool was vibe-coded in a single session with Claude.

We tried a few approaches and kept finding that simpler signals outperformed fancier ones. File overlap and issue refs catch most duplicates. Sometimes the obvious solution is the right one.

License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bepo-0.3.0.tar.gz (10.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

bepo-0.3.0-py3-none-any.whl (11.3 kB view details)

Uploaded Python 3

File details

Details for the file bepo-0.3.0.tar.gz.

File metadata

  • Download URL: bepo-0.3.0.tar.gz
  • Upload date:
  • Size: 10.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.2

File hashes

Hashes for bepo-0.3.0.tar.gz
Algorithm Hash digest
SHA256 1694767146c8f7f909fa1712b85de8c6d3c78ae09258087f2dbe4921f999fe2d
MD5 4ae435bc9cdd05f52136415d6ef90c00
BLAKE2b-256 926d36c66d3f36ec642bf8cd246ca3458f5f08f34253a240d1e89da9b76b5f96

See more details on using hashes here.

File details

Details for the file bepo-0.3.0-py3-none-any.whl.

File metadata

  • Download URL: bepo-0.3.0-py3-none-any.whl
  • Upload date:
  • Size: 11.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.2

File hashes

Hashes for bepo-0.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 b3a5ef9bb34f3421bf343caf6e739b75145e0a5fff732f78b09c0b1b65b2a633
MD5 f3faf7f27856cb177146c918b75306e8
BLAKE2b-256 3b5dfd79c2da1076480be9a40e27f2ccadcf4112e4965b6e3177ef4504988844

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page