pdf-strikethrough-detect 0.9.0

Detect struck-through (deleted) text in PDFs and scanned document images.

pdf-strikethrough-detect

Detect struck-through (deleted) text in PDFs and scanned document images.

Strikethrough detection is a surprisingly unserved niche: most "redline"/diff tools assume clean born-digital PDFs and fall apart on scans — which is the real-world case. pdf-strikethrough-detect handles both, does the hard part (scanned images) with a tiny CPU model, and makes no cloud calls.

import pdf_strikethrough as st

# born-digital PDF — exact, no OCR
for w in st.strikethroughs_in_pdf("contract.pdf"):
    print(w["page"], repr(w["chars"]), "partial" if w["partial"] else "full")

Install

pip install pdf-strikethrough-detect

Pure pip, no system binaries required: the CNN runs on ONNX Runtime (CPU) and the ~318 KB model ships inside the wheel. Extras:

pip install "pdf-strikethrough-detect[markdown]"    # clean_markdown() via pymupdf4llm
pip install "pdf-strikethrough-detect[rapidocr]"    # free scanned-word OCR backend (no binary)
pip install "pdf-strikethrough-detect[tesseract]"   # word-level OCR (also needs the tesseract binary)
pip install "pdf-strikethrough-detect[torch]"       # optional .pt CNN fallback for dev/retraining

The CNN ships as ONNX and needs nothing extra. [torch] is only for development — the loader prefers strike_verdict_cnn.onnx, falling back to a strike_verdict_cnn.pt checkpoint if you point PDF_STRIKETHROUGH_MODEL_DIR at one. Regenerate the shipped ONNX from a trained checkpoint with tools/export_model.py.

Native / born-digital PDFs — exact

In a born-digital PDF a strikethrough is a vector drawing (a line or thin rect over the text), so detection is exact ground truth — no OCR, no model, no guessing. Both vector-rule and filled-rect strikethrough styles are handled.

import pdf_strikethrough as st

for w in st.strikethroughs_in_pdf("contract.pdf"):
    print(w["page"], repr(w["chars"]))          # 'chars' = the struck substring
print(st.clean_markdown("contract.pdf"))        # surviving text, deletions removed (needs [markdown])

Each native record: {page, text, chars, char_span, partial, bbox_frac, tier, coverage, verdict, final} (tier is "vector", "flag", or "annot"). Vector records also carry stroke_color/stroke_width (the paint + thickness of the dominant stroke — red = opposing counsel is evidence); annotation records carry annot_author/annot_created/annot_modified/ annot_color/annot_id (the redline's "who and when"). Scanned records replace coverage with the evidence that decided them — score, cnn_prob, cnn_agrees, conf — and tier is "auto"/"review". The keys are documented as TypedDicts in pdf_strikethrough.types (StruckWord, DetectResult, Passage); the package ships py.typed, so type checkers see them. Partial strikes (semi- of semi-monthly) are resolved to a char range. bbox_frac is in fractions of the rendered page (rotation-aware), so it maps directly onto a rendered pixmap.

Three native detectors, all base-PyMuPDF only (no pymupdf4llm), selected by method:

st.strikethroughs_in_pdf("contract.pdf", method="vector")  # stroke geometry (default) —
                                                           #   precise partial-char spans + color/width
st.strikethroughs_in_pdf("contract.pdf", method="flag")    # MuPDF's FZ_STEXT_STRIKEOUT signal —
                                                           #   also catches font-attribute strikes
st.strikethroughs_in_pdf("contract.pdf", method="annot")   # explicit /StrikeOut annotations —
                                                           #   Acrobat/Preview redlines + forensics
st.strikethroughs_in_pdf("contract.pdf", method="both")    # union of all three — maximum recall

Validated across domains. On 10 public redline PDFs (federal regulatory redlines from the Copyright Office, FDIC, CEQ and EPA; California privacy-regulation redlines; a municipal development code — 54.7k struck words), 99.8% of vector detections are independently confirmed by MuPDF's strikeout signal (92.5–100% per document; 8 of 10 ≥99.9%), and the flag method adds ~2.6% more words (font-attribute strikes and edge cases) — use method="both" to capture them. pymupdf4llm is not used for detection at all; it is only an optional [markdown] extra for richer layout in clean_markdown().

Any PDF — routed per page, scanned pages use OCR + CNN

import pdf_strikethrough as st
from pdf_strikethrough.ocr import rapidocr_backend
from pdf_strikethrough.scanned import ScanConfig

res = st.detect_pdf("mixed.pdf",
                    ocr=rapidocr_backend(),                 # for scanned pages
                    scan_config=ScanConfig.confidence_free())

struck   = [w for w in res["words"] if w["final"]]          # struck words (boxes, char spans)
markdown = res["markdown"]                                  # deletions as ~~struck~~
clean    = res["clean_text"]                                # surviving text, deletions removed
passages = res["passages"]                                  # grouped deletion sections

detect_pdf classifies each page native-vs-scanned, runs the exact path on native pages (method="vector"|"flag"|"annot"|"both") and the geometry→OCR→CNN pipeline on scanned ones, and assembles markdown / clean_text / passages for both page kinds from its own strike decisions (so the text and the word records always agree — no dependence on an external markdown engine). Already have an Azure Document Intelligence result? Pass di_result=... (the REST JSON dict, an {'analyzeResult': ...} envelope, or sdk_result.as_dict()) to skip re-OCR and use DI's word boxes.

Scanned pages with no OCR backend raise OcrRequiredError by default; pass on_missing_ocr="skip" to skip them (with a warning in res["warnings"]) and still get everything from the native pages. Password-protected PDFs raise EncryptedPdfError.

clean_markdown() remains a separate, higher-fidelity native-only path that borrows pymupdf4llm's layout (headings, paragraphs). detect_pdf's markdown is layout-plain but works uniformly on scanned pages too.

Choosing an OCR backend

The geometry + CNN carry the detection and are OCR-independent; OCR only supplies word boxes to attribute strikes to, plus a confidence prior. Benchmarked by rasterizing born-digital redline pages into image-only "scans" and taking the native detector's strikes on the born-digital originals as ground truth (3 documents, 24 pages, 2,170 known strikes; reproduce with benchmarks/scanned_recovery.py):

Backend	Setup	Strike recovery	Word granularity
Azure Document Intelligence	cloud, paid	95% of known strikes	exact word boxes
RapidOCR	pip, no binary	97% of known strikes	~4× coarser (phrase-level)
Tesseract	needs system binary	not benchmarked here	genuine word-level

Use ScanConfig.confidence_free() with RapidOCR (its confidences cluster near 1.0 and don't separate struck from clean text); the default ScanConfig() is calibrated to Azure DI, whose struck words drop to 0.43–0.94. Across backends the scanned path recovers 95–97% of the exact native strike set — the geometry + CNN, not the OCR engine, carry the detection.

Beyond PDFs — images, Word docs, cloud OCR

The same detection reaches inputs that never were a born-digital PDF:

# Raster image files (.png/.jpg/.tiff, incl. multi-page TIFF) — photos, faxes, scans.
# Every frame is a scanned page, so it needs OCR; DPI comes from image metadata (else 200).
res = st.detect_image_file("scan.tiff", ocr=rapidocr_backend())

# Word .docx — strike formatting (w:strike/w:dstrike) + tracked deletions (w:del, with author/date).
# No OCR, no geometry: records use tier="docx" and a `para` index. Stdlib-only, no extra needed.
for w in st.strikethroughs_in_docx("contract.docx"):
    print(w["para"], repr(w["chars"]), w["docx_change"], w.get("docx_author"))

# AWS Textract / Google Document AI — neither flags strikethrough; add the layer on top.
# Convert a pre-fetched result to {page: [Word]} and feed it as words_by_page (works on images too).
res = st.detect_pdf("doc.pdf", words_by_page=st.words_from_textract(textract_response))
res = st.detect_pdf("doc.pdf", words_by_page=st.words_from_docai(docai_document))

Provenance mode for RAG. Struck text entering a vector index is a real failure mode — the retriever surfaces a sentence the document deleted. clean_text removes it; provenance_text instead keeps each deletion as a marker so it's recorded-as-deleted, not silently dropped:

res = st.detect_pdf("contract.pdf")
res["clean_text"]           # 'The fee is billed monthly.'
st.provenance_text(res)     # 'The fee is [deleted: the old rate of 5%] billed monthly.'

Low-level building blocks

gray = st.render_page_gray(doc[0], dpi=200)     # always returns a HxW uint8 grayscale array

lines = st.strike_lines(gray, dpi=200)          # OCR-free stroke geometry (strike/underline/rule)
                                                # pass the dpi the image was rendered/scanned at

# word boxes are PAGE FRACTIONS in [0,1], origin top-left — not pixels
p = st.score_word(gray, (0.12, 0.34, 0.38, 0.36))   # CNN strike probability (0..1)

from pdf_strikethrough.ocr import Word
# detect_scanned_image accepts your own image; RGB / float arrays are coerced to grayscale uint8
recs = st.detect_scanned_image(gray, [Word("foo", (0.12, 0.34, 0.38, 0.36), 0.6)])

# visual overlay — render each struck page with the strikes boxed (red=full, orange=partial)
for pg in st.render_overlay("contract.pdf", dpi=150):
    pg["image"].save(f"overlay-p{pg['page']}.png")     # or st.save_overlays(src, "out/")

Logging. Diagnostics (page routing, native method + record counts, OCR/CNN timings) are emitted at DEBUG under the pdf_strikethrough logger — silent by default (a NullHandler is attached). Opt in with logging.getLogger("pdf_strikethrough").setLevel(logging.DEBUG) and a handler; warnings stay reserved for caller-facing hazards.

CLI

pdf-strikethrough detect contract.pdf                       # native pages (scanned pages are
                                                            #   skipped with a warning)
pdf-strikethrough detect scan.pdf --ocr rapidocr            # include scanned pages
pdf-strikethrough detect scan.tiff --ocr rapidocr           # a raster image file (or multi-page TIFF)
pdf-strikethrough detect contract.docx                      # a Word doc (strike + tracked deletions)
pdf-strikethrough detect scan.pdf --di-result di.json       # use a pre-fetched Azure DI result
pdf-strikethrough detect scan.pdf --textract-result t.json  # ...or AWS Textract / Google DocAI
pdf-strikethrough detect doc.pdf --provenance prov.txt      # deletions kept as [deleted: ...] markers
pdf-strikethrough detect doc.pdf --method both              # max-recall native detection
pdf-strikethrough detect doc.pdf --method annot             # explicit /StrikeOut annotations only
pdf-strikethrough detect doc.pdf --pages 1-5,12             # only these pages (1-based)
pdf-strikethrough detect doc.pdf --json out.json            # full struck words + passages + evidence
pdf-strikethrough detect doc.pdf --json -                   # ...or stream JSON to stdout
pdf-strikethrough detect doc.pdf --clean-text clean.txt     # surviving text, deletions removed
pdf-strikethrough detect doc.pdf --markdown marked.md       # deletions as ~~struck~~
pdf-strikethrough detect doc.pdf --overlay out/             # page images with strikes boxed
pdf-strikethrough detect scan.pdf --ocr rapidocr --dump-crops crops/  # export scored crops to label
pdf-strikethrough detect doc.pdf --fail-if-found            # exit 3 if any strike found (CI gate)
cat doc.pdf | pdf-strikethrough detect -                    # read the PDF from stdin
pdf-strikethrough --version

Batch mode. Pass several files, a directory, or a glob and each is processed independently — JSONL output (one result object per line), --jobs N to spread files across processes, and one bad file never aborts the run (its line carries an error key):

pdf-strikethrough detect ./contracts/ --jsonl out.jsonl        # every document in a directory
pdf-strikethrough detect *.pdf --jobs 4 --jsonl -              # glob, 4 workers, stream to stdout
pdf-strikethrough detect ./contracts/ --fail-if-found          # CI gate over a whole tree (exit 3)

Per-file output flags (--markdown/--overlay/--clean-text/--provenance/cloud results/ --pages) are single-file only; in batch mode use --jsonl.

Other flags: --dpi (raster DPI for scanned pages, default 200; read from image metadata for image files — very-high-dpi input is normalized to 200 internally, see below), --overlay-dpi (overlay render DPI, default 150), --limit (max words in plain output), --jobs / --jsonl (batch), --scan-config auto|azure-di|confidence-free, --markdown, --docai-result. Exit codes: 0 ok, 1 usage/file error (batch: ≥1 file errored), 2 encrypted / OCR required, 3 --fail-if-found matched. Full help: pdf-strikethrough detect -h.

Examples

Runnable, dependency-light scripts are in examples/ — they generate their own sample PDFs (no assets to download) so every snippet is copy-paste runnable:

python examples/native_quickstart.py     # build a redline PDF, detect strikes, print clean text
python examples/scanned_quickstart.py    # rasterize it to a "scan", run OCR + CNN  ([rapidocr])
python examples/overlay_quickstart.py     # print strike evidence + write a before/after overlay
python examples/rag_provenance.py         # clean_text vs provenance_text ([deleted: ...] markers)

How it works

• Native: merged horizontal vector strokes through a word's middle band (excludes under/over- lines); coverage ≥ 50% → struck, partials resolved to a char range.
• Scanned geometry (lines.py): per-angle morphological opening extracts stroke fragments, collinear fragments are stitched, then strict filters (spine fill, stroke run-thickness, angle/length) separate real strikes from bold crossbars and serif-glyph chains.
• CNN (cnn.py, StrikeNet, 79k params): resolves pixel-ambiguous cases — a thin strike over an ascender-less word is pixel-identical to a glyph chain, and only a learned model tells them apart. Ships as ONNX; set PDF_STRIKETHROUGH_MODEL_DIR to use your own weights.
• Attribution (scanned.py): assigns strokes to OCR words with char spans and full/partial resolution, plus a visual-row "orphan" pass for words the detector's stroke evidence missed.

Resolution & resource limits. The geometry tunables and the CNN's crop are calibrated at 200 dpi; scanned rasters above ~300 dpi are downsampled to 200 internally (accuracy-neutral — extra resolution buys nothing but cost). Output boxes are page fractions, so this is invisible to results. A hard raster budget caps a single page at 128 Mpix, so a hostile huge-mediabox page is auto-downsampled with a warning instead of OOMing the process (see SECURITY.md).

Operating points. The CNN's struck/clean decision threshold is an explicit choice, not a buried constant. ScanConfig.recall_first() biases toward catching every deletion (legal / audit review — never miss a struck word); ScanConfig.precision_first() biases toward never dropping live text as if deleted (RAG / indexing). Calibrate the threshold from your own labeled data with pdf_strikethrough.calibration (threshold_for_recall, threshold_for_precision, conformal_threshold for a distribution-free recall floor) and pass it as recall_first(cnn_p_hi=…).

Improving the model. --dump-crops DIR (or st.dump_crops) exports every crop the pipeline scored plus its verdict as a labeling set; label it, retrain with training/train_strikenet.py, and load your weights via PDF_STRIKETHROUGH_MODEL_DIR, cnn.set_model_dir, or cnn.ensure_model(url, sha256, …) (digest-verified download). A "contribute a failing page" report feeds the same loop, and demo/ is a drag-and-drop Gradio app for trying it — live at huggingface.co/spaces/niles-liu/strikethrough-demo.

Hosted weights. The shipped StrikeNet is also published on the Hugging Face Hub (niles-liu/strikenet); the digest is verified before the graph is ever loaded, so a tampered host can't swap the model:

import pdf_strikethrough as st
st.ensure_model(
    "https://huggingface.co/niles-liu/strikenet/resolve/main/strike_verdict_cnn.onnx",
    "fac2c51baaa75ee782196bdfe7452638cb48c7deddb21163b1ac6a0a72ae4457",
    meta_url="https://huggingface.co/niles-liu/strikenet/resolve/main/strike_verdict_cnn.meta.json",
)

Scope — validated scripts & layouts. Detection assumes horizontal, left-to-right text; a strike is a near-horizontal stroke through a word's middle band. Reading-order assembly is column-aware (two-column pages read down each column; tables read across rows).

Case	Status
Latin / Western, horizontal	validated (10-PDF redline benchmark, 54.7k struck words)
CJK, horizontal set (native + scanned)	detected — strikes are horizontal regardless of script (regression-tested)
Two-column / newspaper layout, struck table rows, hyphenated-line-break strikes	handled
Vertical writing modes (CJK/Mongolian), RTL strike axes	out of scope (roadmap)
Handwritten / freehand strikes	untested (geometry assumes near-straight strokes)

RapidOCR (the recommended scanned backend) is strongest on Chinese, so horizontal-CJK scans work end to end today; full vertical-text support is future work.

Handwritten strikes are the main known gap: the geometry gate looks for a near-straight, near-horizontal stroke, and the CNN was trained on rendered digital strikes, so a wavy pen scribble or a heavy cross-out may be missed. If you hit one, a failing-page report with --dump-crops output is the fastest way to turn it into a fix — it becomes both a regression test and training data.

API surface

The top-level package (import pdf_strikethrough as st) exports ~37 names; the most-used:

Group	Names
High-level	strikethroughs_in_pdf, clean_markdown, provenance_text, detect_pdf, detect_image_file, detect_scanned_image, strikethroughs_in_docx, open_pdf, render_page_gray, render_overlay, save_overlays
Native detectors	native_page_strikes, native_flag_strikes, native_annot_strikes, native_doc_strikes, page_strikes, native_markdown, strip_struck_markdown
Scanned geometry + classifier	strike_lines, ink_mask, to_gray_u8, analyze_scanned_page, ScanConfig, classify_page_source, apply_cnn_verdict
CNN	score_word, score_crops, std_crop, word_crop_px, verdict_of, get_model_meta, ensure_model
Active learning + calibration	dump_crops, pdf_strikethrough.calibration (threshold_for_recall, threshold_for_precision, conformal_threshold, pr_curve)
OCR (backends + cloud-result adapters)	Word, rapidocr_backend, tesseract_backend, words_from_azure_di, words_from_textract, words_from_docai
Errors	OcrRequiredError, EncryptedPdfError
Types	StruckWord, DetectResult, Passage (in pdf_strikethrough.types)

Everything is docstringed; help(st.detect_pdf) is the reference.

Contributing, security, citation

• Development setup, running the tests, and regenerating the model: CONTRIBUTING.md.
• This package parses untrusted PDFs — reporting policy in SECURITY.md.
• Citing it in research: CITATION.cff.

License

MIT.