PrettyPipeline

Turn PDFs into structured JSON. Text comes from embedded PDF content or local OCR. Figures are segregated and sent directly to GPT-5.4 — never the whole PDF as one image. Export to CSV and common LLM fine-tuning formats.

Architecture

                         ┌─────────────────────────────────────┐
                         │              PDF input              │
                         └─────────────────┬───────────────────┘
                                           │
              ┌────────────────────────────┼────────────────────────────┐
              │                            │                            │
              ▼                            ▼                            ▼
     ┌────────────────┐         ┌──────────────────┐         ┌──────────────────┐
     │ Embedded text  │         │ Embedded figures │         │ Scan-only pages  │
     │ (digital PDF)  │         │ (PyMuPDF crop)   │         │ (low-res fallback│
     └───────┬────────┘         └────────┬─────────┘         │  120 DPI max)    │
             │                           │                   └────────┬─────────┘
             │                           │                            │
             │              ┌────────────┴────────────┐               │
             │              │  NOT full-page images   │               │
             │              │  detail=low by default  │               │
             │              └────────────┬────────────┘               │
             │                           │                            │
             └───────────────────────────┼────────────────────────────┘
                                         │
                                         ▼
                            ┌────────────────────────┐
                            │   GPT-5.4 nano (cloud) │
                            │   text + figure images │
                            │   strict JSON schema   │
                            └───────────┬────────────┘
                                        │
                    ┌───────────────────┼───────────────────┐
                    ▼                   ▼                   ▼
              ┌──────────┐      ┌─────────────┐    ┌──────────────┐
              │   JSON   │      │ needs_review│    │ Export formats│
              │  + _meta │      │   flags     │    │ csv alpaca   │
              └──────────┘      └─────────────┘    │ sharegpt qa  │
                                                   │ openai jsonl │
                                                   └──────────────┘

Local OCR (Unlimited-OCR) only when a page has no usable embedded text
and `--force-ocr` is set or hybrid routing needs it. Requires `[ocr]` extra.

Why this is cheap

• Digital PDFs — embedded text is free and accurate; no GPU needed.
• Figures only — segregated embedded images go to GPT vision at detail=low (not full-page raster).
• Local OCR — baidu/Unlimited-OCR for scan pages only (pip install prettypipeline-ocr[ocr]).
• GPT-5.4 nano — ~$0.20 / 1M input tokens for structuring (pricing).

Install

Python 3.10–3.13 (3.12 recommended).

# Digital PDFs + GPT vision (smallest install)
pip install prettypipeline-ocr

# + local OCR for scans
pip install prettypipeline-ocr[ocr]

From this repo:

python3.12 -m venv .venv && source .venv/bin/activate
pip install -e ".[ocr,dev]"

NVIDIA Linux:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124
pip install prettypipeline-ocr[ocr]

Docker (CUDA): see Dockerfile.cuda.

Quick start

export OPENAI_API_KEY=sk-...

# Extract JSON (text + segregated figures → GPT)
prettypipeline run invoice.pdf --schema examples/invoice.schema.json

# Also export fine-tuning formats
prettypipeline run invoice.pdf --schema examples/invoice.schema.json \
  -o out.json --export csv,alpaca,sharegpt,openai,qa

# Convert existing JSON to a training file
prettypipeline export out.json --schema examples/invoice.schema.json \
  --format sharegpt -o train.jsonl

Demo invoice:

prettypipeline run tests/fixtures/sample_invoice.pdf \
  --schema examples/invoice.schema.json -o out.json

Figure segregation (vision)

PrettyPipeline does not rasterize entire PDF pages for GPT unless a page is scan-only with no embedded figures.

Images are visual context only — no CSV or table dump is generated from figures.

Export formats (LLM training)

ShareGPT and OpenAI JSONL are preferred for chat models in 2026; Alpaca for single-turn extraction tasks.

Schema examples

CLI

prettypipeline run FILE.pdf --schema SCHEMA.json
  -o, --output PATH           JSON output (also printed)
  --export csv,alpaca,...     comma-separated training exports
  --no-vision                 skip figure images to GPT
  --image-detail low|auto|high  vision token budget (default: low)
  --ocr-only                  skip GPT step
  --force-ocr                 always OCR text locally
  --model, --base-url         LLM settings

prettypipeline batch FILES... --schema SCHEMA.json --output-dir DIR
prettypipeline export RESULT.json --schema SCHEMA.json --format FORMAT -o OUT

Environment: OPENAI_API_KEY, PRETTYPIPELINE_MODEL, PRETTYPIPELINE_BASE_URL.

Output

{
  "data": { "invoice_number": "INV-3337", "total": 93.5 },
  "needs_review": [],
  "source": "pdf_text",
  "_meta": {
    "version": "0.4.0",
    "elapsed_ms": 3200,
    "pages": 1,
    "vision_images": 1,
    "segments": { "text_chars": 842, "figures": 1 },
    "token_usage": { "prompt_tokens": 1100, "completion_tokens": 95, "total_tokens": 1195 }
  }
}

Programmatic use

from prettypipeline import run, segment_pdf, to_csv

result = run("invoice.pdf", "examples/invoice.schema.json", export_formats=["alpaca"], export_stem=Path("out.json"))
seg = segment_pdf("invoice.pdf")  # inspect text vs figures before calling GPT

Hardware

Development

pip install -e ".[ocr,dev]"
pytest
python -m build && twine check dist/*

License

MIT. Unlimited-OCR is also MIT.

Fixtures: Sliced Invoices sample, IRS Form W-9.