Skip to main content

Detect duplicate PRs in GitHub repos

Project description

bepo

Detect duplicate pull requests in GitHub repos.

No ML, no embeddings, no API keys. Just static analysis of diffs.

A maintainer with 100 open PRs can run bepo check --repo foo/bar and in 5 minutes get a ranked list of "you should look at these pairs." That saves hours of manual review.

The Problem

Large repos waste engineering time on duplicate PRs. When multiple contributors fix the same bug independently, only one PR gets merged — the rest is wasted effort.

This actually happens. We analyzed 100 PRs from OpenClaw and found:

Cluster PRs What happened
Matrix startup bug 4 PRs 4 engineers independently fixed startupGraceMs = 05000
Media token regex 2 PRs Identical fix submitted twice
Feishu bitable config 2 PRs Same multi-account config fix

8 duplicate PRs across 3 bug fixes. That's real engineering time wasted.

Proof: OpenClaw Analysis

We ran bepo on OpenClaw's open PRs. Here's what it found:

$ bepo check --repo openclaw/openclaw --limit 100

#20025 <-> #19973
  Similarity: 86%
  Reason: Both fix #19843      ← Same issue!

#19868 <-> #19855
  Similarity: 81%
  Reason: Same files: parse.ts, pi-embedded-subscribe.tools.ts

#19871 <-> #19853
  Similarity: 100%
  Reason: Same files: bitable.ts, config-schema.ts, tools-config.ts

Verified manually:

PR Pair Similarity Verdict
#20025 ↔ #19973 (Matrix) 86% ✅ TRUE DUPLICATE — both change startupGraceMs from 0 to 5000
#19868 ↔ #19855 (regex) 81% ✅ TRUE DUPLICATE — identical PR titles
#19871 ↔ #19853 (Feishu) 100% ✅ TRUE DUPLICATE — same files, same fix
#19996 ↔ #19993 (unrelated) 20% ✅ Correctly NOT flagged

Precision: 80% (4/5 flagged clusters were true duplicates)

More Examples

VSCode — Found PRs touching same files for same feature:

#295823 <-> #295822
  Similarity: 77%
  Reason: Same files: chatModel.ts, chatForkActions.ts

  Both: "Use metadata flag for fork detection"

Next.js — Found related test updates:

#90121 <-> #90120
  Similarity: 86%
  Reason: Same files: test/

Install

pip install bepo

Requires GitHub CLI (gh) to be installed and authenticated.

Usage

# Check a repo for duplicate PRs
bepo check --repo owner/repo

# Adjust sensitivity (default: 0.4, higher = stricter)
bepo check --repo owner/repo --threshold 0.5

# Check more PRs
bepo check --repo owner/repo --limit 100

# JSON output for CI
bepo check --repo owner/repo --json

How It Works

bepo fingerprints each PR by extracting:

Signal Weight What it catches
Same issue ref (#123) 10.0 Definite duplicate
Same code changes (IDF-weighted) 8.0 Rare lines weighted more than common boilerplate
Same files touched 6.0 PRs modifying same code
Same feature domain 3.0 auth, messaging, database, etc.
Same imports 1.0 Similar dependencies

Then computes pairwise Jaccard similarity.

That's it. No embeddings, no LLM calls. Just:

  • Parse +++ b/path from diffs
  • Regex for #\d+ issue refs
  • Compare actual code changes
  • Set intersection for similarity

~300 lines of Python.

As a Library

from bepo import fingerprint_pr, find_duplicates

# Fingerprint PRs
fp1 = fingerprint_pr("#123", diff1, title="Fix auth", body="Fixes #456")
fp2 = fingerprint_pr("#124", diff2, title="Auth fix", body="Fixes #456")

# Find duplicates
dups = find_duplicates([fp1, fp2], threshold=0.4)
for d in dups:
    print(f"{d.pr_a}{d.pr_b}: {d.similarity:.0%}")
    print(f"  Shared issues: {d.shared_issues}")
    print(f"  Shared files: {d.shared_files}")

GitHub Action

Add to your repo to automatically detect duplicate PRs and post a warning comment:

name: PR Duplicate Check
on: [pull_request]

jobs:
  check-duplicates:
    runs-on: ubuntu-latest
    steps:
      - uses: aardpark/bepo@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          threshold: '0.5'  # optional, default 0.5

When a PR is opened that looks like a duplicate, bepo posts a comment:

⚠️ Potential Duplicate PRs Detected

This PR may be similar to existing open PRs:

PR Similarity Reason
#123 85% Both fix #456
#124 71% Same code: 10 lines overlap

Detected by bepo

Action Inputs

Input Description Default
github-token GitHub token for API access ${{ github.token }}
threshold Similarity threshold (0.0-1.0) 0.5
limit Max PRs to compare against 50
comment Post comment on PR true

Action Outputs

Output Description
has_duplicates true if duplicates found
match_count Number of matches
matches JSON array of matches

Why This Works

Duplicates share obvious signals:

  • Same code = Identical changes (639 shared lines caught SoundChain duplicates)
  • Same issue ref = Same bug report (#19843 appeared in 4 Matrix PRs)
  • Same files = Same bug location (100% overlap for Feishu cluster)

IDF weighting makes rare lines matter more than common boilerplate. A shared startupGraceMs = 5000 is a stronger signal than a shared return null.

Code overlap and issue refs catch most duplicates. Simple works.

Origin Story

This tool was vibe-coded in a single session with Claude.

We tried a few approaches and kept finding that simpler signals outperformed fancier ones. File overlap and issue refs catch most duplicates. Sometimes the obvious solution is the right one.

License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bepo-0.6.0.tar.gz (13.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

bepo-0.6.0-py3-none-any.whl (12.7 kB view details)

Uploaded Python 3

File details

Details for the file bepo-0.6.0.tar.gz.

File metadata

  • Download URL: bepo-0.6.0.tar.gz
  • Upload date:
  • Size: 13.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.2

File hashes

Hashes for bepo-0.6.0.tar.gz
Algorithm Hash digest
SHA256 3f863b181148a4626fe1cc9f7a74166d9ee77fd76666220b3f25fc27404ebad0
MD5 d003fb3bc6937e8316d5ed0b28ff44b9
BLAKE2b-256 c26f9e232730fd251fb01dd365a8720e5d399c2b516541497bc0bc4af1ba28dc

See more details on using hashes here.

File details

Details for the file bepo-0.6.0-py3-none-any.whl.

File metadata

  • Download URL: bepo-0.6.0-py3-none-any.whl
  • Upload date:
  • Size: 12.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.2

File hashes

Hashes for bepo-0.6.0-py3-none-any.whl
Algorithm Hash digest
SHA256 dab6a092960d3c8e5b6bf3cfc5a3b24e1e02fcf09e6ac124f01c1491cb675ec3
MD5 98a2d68ce444dc10e88dc07b406fe042
BLAKE2b-256 94089ac6687d8f5d9389285621f6f437356e68c228ebf8fbea43292b0a62bceb

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page