Wikipedia Gap Finder v2

A campaign-driven pipeline that finds people from an underrepresented group who should have an English Wikipedia article but don't, checks every fact against real sources, and hands a human a research dossier they write the article from.

The tool never writes article prose. English Wikipedia prohibits LLM-generated/-rewritten article content (guideline enacted by RfC, March 2026, with narrow copyedit and translation exceptions). This tool does detective work, verification, and structure only — a human writes every sentence, and the dossier is a map, not a source: the writer opens and reads every cited source themselves. That isn't a limitation to route around; it's the product's integrity claim: AI found her and gathered the evidence; a human checked every source and wrote her story.

Pipeline

1. Intake — names from CSV/txt lists and/or an optional Wikidata SPARQL redlist query.
2. Gap check — per name: enwiki title + fuzzy search, redirect detection, Draft: namespace, deletion-log history, and Wikidata sitelinks. Verdicts: GAP / EXISTS / REDIRECT_ONLY / DRAFT_EXISTS / DELETED_BEFORE / TRANSLATE_CANDIDATE (⭐ candidates that already exist in another language are the highest-value, since translation is an allowed path).
3. Notability triage (planned) — coverage search scored against WP:GNG.
4. Source vetting → dossier — see below.

Source vetter

The vetter is two processes across a two-file JSON boundary, so each side is testable in isolation:

• Python (keyless): gathers coverage, tiers each source against Wikipedia's perennial-sources list (WP:RSP), and renders the dossier.

• The Claude subagent (skills/vet-sources): classifies reliability and traces breadcrumbs, sorting every claim into one of three buckets:

  Bucket	Meaning	Dossier section
  VERIFIED	a single reliable source names the subject and supports the claim, with a verbatim quote	Verified facts table
  LEAD	a breadcrumb is real, but no reliable source names the subject yet	Research leads — chase before writing
  DEAD_END	no corroboration found	UNVERIFIED — do not use

The never-stitch rule (WP:SYNTH): the vetter may not promote a LEAD to VERIFIED by combining a subject-naming unreliable source with a fact-confirming reliable source. Two half-sources stay a LEAD. And the renderer structurally refuses to print any fact backed only by an unreliable source — even if a verdicts file claims otherwise.

Install

pip install wiki-gap-finder            # installs the `gap-finder` command
pip install "wiki-gap-finder[ui]"      # + Rich-rendered tables
pip install "wiki-gap-finder[chase]"   # + the Firecrawl coverage backend

Or clone this repo and run python3 gap_finder.py ... directly — same CLI (pip3 install requests PyYAML jsonschema --break-system-packages).

Usage

Commands read and write paths relative to the current directory, so run them from your project root (campaign paths like input/candidates.csv resolve from there, and results land in output/<campaign>/).

gap-finder --campaign campaigns/<campaign>.yaml --check       # stages 1–2
gap-finder --campaign campaigns/<campaign>.yaml --check --no-sparql --limit 5
gap-finder --campaign campaigns/<campaign>.yaml --report      # table from saved state
gap-finder --campaign campaigns/<campaign>.yaml --dossier "Name Here"
gap-finder --campaign ... --dossier "Name" --search-backend firecrawl

A campaign is one small YAML — swapping demographics is a new file, zero code changes. One rule for lists naming living people (the spirit of WP:BLPCAT): only include identity attributes the person has publicly self-identified with, and only when they're relevant to their public life. Minimal example:

name: my-campaign
description: Who this campaign is about
intake:
  name_lists:
    - "input/candidates.csv"   # CSV with a `name` column, or one name per line
  # optional: a Wikidata SPARQL WHERE-clause body for redlist intake
search_hints:                  # appended to coverage searches
  - "activist"

Other flags: -v logs every request.

Heads-up on the keyless search default: DuckDuckGo increasingly serves a bot challenge to non-browser clients. If coverage search comes back empty, the tool now says so loudly — use --search-backend firecrawl (with the firecrawl CLI installed) for dependable coverage gathering.

Conventions

Zero API keys on the default path (MediaWiki + Wikidata + DuckDuckGo are keyless; Firecrawl is opt-in). Polite User-Agent, ≥1s between requests with exponential backoff, and maxlag=5 on MediaWiki API calls so the tool yields when servers lag.

python3 -m pytest      # 54 tests

Layout

• gap_finder.py — back-compat shim; the CLI lives in gapfinder/cli.py
• gapfinder/ — the package (cli, contract, rsp, search, worklist, verdicts, dossier)
• gapfinder/data/rsp_seed.json — curated WP:RSP reliability seed (ships in the wheel)
• skills/vet-sources/SKILL.md — the Claude subagent contract
• campaigns/ — one YAML per campaign
• docs/superpowers/ — design spec and implementation plan

MIT licensed.