VeriScript

Offline document restoration. A photo, scan or PDF goes in; a cleaned page with a searchable PDF, overlay, transcript, Markdown and OCR JSON comes out — with honest flags on the numbers it is not sure about. Devanagari (Nepali/Hindi) first, English supported by the same pipeline.

Everything runs on your machine. No cloud calls, no telemetry, no account.

Live demo: https://veriscript.live — one page at a time, no signup; demo uploads are deleted after the 60-minute retention window.

photo/scan/PDF ──▶ orientation ──▶ layout ──▶ OCR ──▶ risk flags ──▶ searchable PDF
                                                                  ├─ overlay.png (review boxes)
                                                                  ├─ transcript.txt
                                                                  ├─ transcript.md (Markdown)
                                                                  └─ ocr.json (tokens + queue)

Before / after

Real pipeline output. Green = accepted, amber = uncertain, red = digit conflict with the alternative reading kept. The letterpress page is from heiDATA (doi:10.11588/data/EGOKEI, CC BY 4.0); the photo row is this repository's own synthetic fixture under a simulated phone shadow. Regenerate with python scripts/make_demo_image.py.

Why it exists

• It will not invent the numbers on your bill. Digits are the highest-risk tokens, so they are re-read, risk-ranked and put in a review queue instead of being silently guessed. The queue is measured, not asserted (EVALUATION.md).
• Devanagari is a first-class citizen, not a language pack. Letterpress books, modern government PDFs and phone photos each have frozen evaluation sets with published numbers and confidence intervals.
• Offline by default. Privacy is the reason to use this instead of a cloud lens: the pipeline runs with networking disabled.

Install

From a source checkout (the package is not on PyPI yet):

pip install -r requirements.txt        # full runtime (includes torch)
pip install -r requirements-dev.txt    # + dev/eval tooling (optional)

Or install the Python package itself — the document pipeline is torch-free (photo/verifier are opt-in extras):

pip install -e .                  # document pipeline + web/CLI
pip install -e ".[photo]"         # + photo upscaling (torch)
pip install -e ".[verifier]"      # + the optional bodhan digit verifier

Optional: Tesseract 5 as a second OCR backend (clean-scan fallback and the OSD rotation fallback).

Use

CLI

# Photo/scan in: searchable PDF + overlay + transcript + Markdown + JSON out
python cli.py --mode document --input page.jpg --output out/run1

# PDF in: first page by default; --max-pages 3 = first three pages
python cli.py --mode document --input scan.pdf --output out/run1 --max-pages 3

# PDF → Markdown: per-page .md always; --max-pages 0 adds one combined .md
python cli.py --mode document --input report.pdf --output out/report --max-pages 0

# Same CLI as a module / installed console script:
python -m veriscript --mode document --input report.pdf --output out/report

# Photo upscaling (shares the same CLI/app)
python cli.py --mode photo --input photos/ --output out/upscaled --scale 4

Useful knobs (all default to the measured best configuration):

--ocr rapidocr|tesseract     --lang en|ne|hi       --deskew
--no-reading-order           --rotate auto|off     --repass-digits/--no-repass-digits
--digit-verifier off|bodhan  (optional second-model digit check)
--no-pdf --no-overlay --no-txt --no-md

Web app (the studio)

The single UI — FastAPI + SolidJS, running the same pipeline and outputs (searchable PDF, transcript, Markdown, OCR JSON) as the CLI.

pip install -r webapp/requirements-web.txt    # backend, on top of requirements.txt
cd webapp/frontend && npm install && npm run build && cd ../..
python webapp/server.py                       # http://127.0.0.1:8000

Hosted deployments (Docker, basic auth, HF Spaces/Render/Fly) are covered in docs/DEPLOY.md. The server binds to localhost by default; the pipeline, models and any training artifacts stay server-side. The photo upscaler is CLI-only (its weights are non-commercial); the web studio is document-only — see docs/PLAN_WEB_FULL.md.

For JS/TS integrations there is a typed HTTP client in packages/veriscript-client (npm: veriscript-client; not published yet).

Outputs (per page)

For an input page.jpg, every run writes:

The web studio serves the same artifacts under canonical names (restored.png, searchable.pdf, transcript.txt, transcript.md, ocr.json, combined.md). --no-md (CLI) or the Markdown toggle (web) skips the .md files.

Measured behaviour

Numbers below come from hash-frozen evaluation sets with bootstrap 95% CIs (full tables, provenance and rebuild commands: docs/EVALUATION.md). No number in this repo is marketing.

Known limitations are listed with their evidence in docs/EVALUATION.md and the full research log in docs/PLAN.md. Since 2026-10-01 the image paths (CLI and web studio) default to the auto line-reader gate — running text + aged paper only. PDF inputs keep the engine reading, and the hosted document-only demo ships no reader checkpoint, so it stays on the engine there.

How it compares

Head-to-head on the frozen Devanagari hard-10 (the table-heavy hardest 10 of the 41-page nepali_pdf_v2 set, Gemini-2.5-Pro anchor GT; every arm scored by the same metric code — canonical write-up in docs/BENCHMARK.md, raw lab notes and reproduction commands in evals/bakeoff_results.md):

Speeds are per page on the harness hardware — ours, raw and Surya on an RTX 2050 4 GB (Surya via llama.cpp/Vulkan), Qwen on a T4 (NF4), PaddleOCR on CPU. Runs on CPU: ✓ = CPU-first (PaddleOCR was measured CPU-only here; VeriScript and RapidOCR ship CPU-first); ✗ = GPU-class in practice — a GPU was used and CPU would be far slower.

Metrics. CER (character error rate) is the edit distance between the OCR output and the ground truth (insertions + deletions + substitutions) divided by the number of ground-truth characters: order-sensitive, 0 is perfect, lower is better. bagCER is the same character distance after alphabetically sorting the tokens on both sides — it ignores reading order, so it isolates glyph recognition from layout (a page with perfect words in the wrong order still scores 0). Median CER is the middle page's CER, robust to a single catastrophic page; s/page is seconds per page.

Reading. bagCER is tied across every arm (0.360–0.400): glyph recognition is a commodity, and the page-CER spread is reading order, table structure and digit handling — which is where this pipeline earns its keep (0.530 → 0.255 over the raw engine on the same frozen pages, while staying CPU-first, offline and Apache-2.0). Surya 2 reads best by median at ~23× our page time and OpenRAIL-M weights, and the GPU arms invent text the source never contained (82–224 tokens over the 10 pages); only this stack ships an alternative-reading digit review queue.

Project layout

veriscript/               the product package
  cli.py                  command-line entry point (document + photo)
  core/                   dataset IO + metrics (doc_data, doc_metrics), degradation
  document/               pipeline: OCR, layout, orientation, restore, export, router, verifier
  deva/                   Devanagari line reader (CRNN+CTC), opt-in via --deva-lines
  photo/                  photo/vector stack (parked; CLI photo mode)
  branding.py, logging_setup.py, lexicon.py, calibration.py
cli.py                    compatibility launcher for `python -m veriscript`
webapp/                   the studio: FastAPI backend (vvweb/) + SolidJS frontend
deva_crnn/                training package for the line reader (repo tooling)
evals/harness/            evaluation harnesses (frozen sets, metrics, gates)
evals/manifests/          content-hash frozen evaluation sets
scripts/                  data acquisition, training export, gates, release tooling
tests/                    the test suite (pytest counts them; ~338)
docs/                     architecture, evaluation, deployment, licensing, plan

Docs

Tests

python -m pytest tests/ -q

The suite (≈336 collected) covers: OCR/export/routing/CLI/web, layout, orientation, language plumbing, frozen-manifest and CI guards, GT-validity audits, error taxonomy, anchor harness, queue metrics, Unicode-path IO, Devanagari line synthesis, CRNN plumbing (height/width round-trip, beam search, cosine fine-tune), the real-line mining gate, the opt-in line reader (merge geometry, table-rule blocking, auto policy, graceful degradation, provenance), the lexicon fetch/build + unknown_word flag mechanics, the textbook harvest parsers + real-print scan probe, the CC-100 corpus builder + controlled sampler, the Vertex wheel builder and the trainer GCS output path, the Devanagari PDF text-layer extraction round-trips, the logging configuration + server-error capture, and the Markdown export (per page and combined).

Data, models and licensing

This repository contains code and evaluation evidence only: no training datasets, no model checkpoints and no redistributed third-party corpora. Evaluation manifests record hashes and provenance; the data itself stays local (data/, weights/ and artifacts/ are git-ignored). The optional Devanagari lexicon behind the unknown_word flag is built locally by scripts/fetch_nepali_lexicon.py (Apache-2.0 + MIT sources) and is not redistributed. Third-party license obligations and the air-gap story are tracked in docs/LICENSES.md.

MIT licensed — see LICENSE. Third-party components keep their own licenses (RapidOCR/PP-OCR Apache-2.0, reportlab/pypdfium2 permissive, PySide6 LGPL, UltraSharp weights CC-BY-NC-SA and therefore excluded from commercial builds).