PrettyPipeline

Turn PDFs into structured JSON. Text comes from embedded PDF content or local OCR. Figures are segregated and sent directly to GPT-5.4 — never the whole PDF as one image. Export to CSV and common LLM fine-tuning formats.

Architecture

                         ┌─────────────────────────────────────┐
                         │              PDF input              │
                         └─────────────────┬───────────────────┘
                                           │
              ┌────────────────────────────┼────────────────────────────┐
              │                            │                            │
              ▼                            ▼                            ▼
     ┌────────────────┐         ┌──────────────────┐         ┌──────────────────┐
     │ Embedded text  │         │ Embedded figures │         │ Scan-only pages  │
     │ (digital PDF)  │         │ (PyMuPDF crop)   │         │ (low-res fallback│
     └───────┬────────┘         └────────┬─────────┘         │  120 DPI max)    │
             │                           │                   └────────┬─────────┘
             │                           │                            │
             │              ┌────────────┴────────────┐               │
             │              │  NOT full-page images   │               │
             │              │  detail=low by default  │               │
             │              └────────────┬────────────┘               │
             │                           │                            │
             └───────────────────────────┼────────────────────────────┘
                                         │
                                         ▼
              ┌────────────────────────┐
              │   Embedded PDF text    │
              └───────────┬────────────┘
                          │ merge (dedupe)
              ┌───────────▼────────────┐
              │ Baidu Unlimited-OCR    │  ← default: always runs ([ocr] extra)
              │ (full document)        │
              └───────────┬────────────┘
                          │
                          ▼
              ┌────────────────────────┐
              │  Combined document text │
              │  + segregated figures   │──► GPT-5.4 nano
              └────────────────────────┘
                          │
          ┌───────────────┼───────────────┐
          ▼               ▼               ▼
     ┌─────────┐   ┌─────────────┐  ┌──────────────┐
     │  JSON   │   │needs_review │  │Export formats│
     │ + _meta │   │   flags     │  │ csv alpaca…  │
     └─────────┘   └─────────────┘  └──────────────┘

Local OCR (Unlimited-OCR) runs on every document by default (requires [ocr] extra), merged with embedded PDF text so nothing is missed — text in images, stamps, scan layers. Use --embedded-only to skip OCR (faster). Use --force-ocr for OCR-only.

Why this is cheap

• Digital PDFs — embedded text is free and accurate; no GPU needed.
• Figures only — segregated embedded images go to GPT vision at detail=low (not full-page raster).
• Baidu OCR + merge — full-document OCR merged with embedded text by default ([ocr] extra).
• GPT-5.4 nano — ~$0.20 / 1M input tokens for structuring (pricing).

Install

Python 3.10–3.13 (3.12 recommended).

# Digital PDFs + GPT vision (smallest install)
pip install prettypipeline-ocr

# + local OCR for scans
pip install prettypipeline-ocr[ocr]

From this repo:

python3.12 -m venv .venv && source .venv/bin/activate
pip install -e ".[ocr,dev]"

NVIDIA Linux:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124
pip install prettypipeline-ocr[ocr]

Docker (CUDA): see Dockerfile.cuda.

Quick start

export OPENAI_API_KEY=sk-...

# Extract JSON (text + segregated figures → GPT)
prettypipeline run invoice.pdf --schema examples/invoice.schema.json

# Also export fine-tuning formats
prettypipeline run invoice.pdf --schema examples/invoice.schema.json \
  -o out.json --export csv,alpaca,sharegpt,openai,qa

# Convert existing JSON to a training file
prettypipeline export out.json --schema examples/invoice.schema.json \
  --format sharegpt -o train.jsonl

Demo invoice:

prettypipeline run tests/fixtures/sample_invoice.pdf \
  --schema examples/invoice.schema.json -o out.json

Figure segregation (vision)

PrettyPipeline does not rasterize entire PDF pages for GPT unless a page is scan-only with no embedded figures.

Images are visual context only — no CSV or table dump is generated from figures.

Export formats (LLM training)

ShareGPT and OpenAI JSONL are preferred for chat models in 2026; Alpaca for single-turn extraction tasks.

Schema examples

CLI

prettypipeline run FILE.pdf --schema SCHEMA.json
  -o, --output PATH           JSON output (also printed)
  --export csv,alpaca,...     comma-separated training exports
  --no-vision                 skip figure images to GPT
  --image-detail low|auto|high  vision token budget (default: low)
  --embedded-only             skip Baidu OCR supplement (embedded text only)
  --force-ocr                 OCR only — ignore embedded text
  --model, --base-url         LLM settings

prettypipeline batch FILES... --schema SCHEMA.json --output-dir DIR
prettypipeline export RESULT.json --schema SCHEMA.json --format FORMAT -o OUT

Environment: OPENAI_API_KEY, PRETTYPIPELINE_MODEL, PRETTYPIPELINE_BASE_URL.

Output

{
  "data": { "invoice_number": "INV-3337", "total": 93.5 },
  "needs_review": [],
  "source": "pdf_text",
  "_meta": {
    "version": "0.4.1",
    "elapsed_ms": 3200,
    "pages": 1,
    "vision_images": 1,
    "segments": { "text_chars": 842, "figures": 1 },
    "token_usage": { "prompt_tokens": 1100, "completion_tokens": 95, "total_tokens": 1195 }
  }
}

Programmatic use

from prettypipeline import run, segment_pdf, to_csv

result = run("invoice.pdf", "examples/invoice.schema.json", export_formats=["alpaca"], export_stem=Path("out.json"))
seg = segment_pdf("invoice.pdf")  # inspect text vs figures before calling GPT

Hardware

Development

pip install -e ".[ocr,dev]"
pytest
python -m build && twine check dist/*

License

MIT. Unlimited-OCR is also MIT.

Fixtures: Sliced Invoices sample, IRS Form W-9.