cheap-ocr 0.0.1

Cheap, fast single-GPU PDF OCR (PP-DocLayoutV3 + GLM-OCR via vLLM)

cheap-ocr

cheap-ocr was born from the need to OCR a large corpus of PDF documents with complex layouts quickly and at low cost. It converts PDF documents to markdown and json, including documents with complex layouts and tables. It uses pp-doclayout-v3 for layout detection and GLM-OCR to recognize text. The throughput is 20 pages per second on our internal benchmarks. It runs on a single L40S GPU (or similar).

Benchmarks

We use 100 PDF documents from the European Medicines Agency to benchmark throughput. The documents are typical business documents with a mix of tables, text, and some figures. All metrics are reported end-to-end including download and upload. Warm-up (model loading and initialization) is excluded from our measurements. We compare cheap-ocr against the official GLM-OCR SDK. The benchmarks ran on a single L40S for both solutions.

Metric	GLM-OCR SDK	cheap-ocr
Pages/sec	5.29	14.26
Output tokens/sec	4,316	11,801
Page regions/sec	75	205.4

Features

Feature	Description
Fast & cheap	Fastest and cheapest VLM-based OCR that we know of.
VLM-based OCR	State-of-the-art text recognition that handles difficult text extraction well.
Layout detection	Recognizes bounding boxes for headings, paragraphs, images, tables, and more.
OCR from cloud storage	Processes PDFs from and writes outputs to S3 buckets, Azure storage containers, or Google Cloud buckets.
One-line deployment	Runs the full pipeline on Modal with a single CLI command.
Bare-metal deployment	Runs on any machine with an NVIDIA GPU.

Use on Modal (easiest, scroll down for bare metal use)

Install cheap-ocr and log into Modal or register a new account.

uv install cheap-ocr[modal,gpu]
modal login

Provide a source folder with PDFs and a target folder where you want to store the outputs. Use the --limit parameter to limit the number of PDFs to process.

# run with local files
# startup takes a few minutes because models need to be loaded and CUDA graphs compiled
uv run cheap-ocr modal --source input_folder/with_pdfs/ --target output_folder/for_results/

# to run cloud storage to cloud cloud storage, install cheap-ocr[s3,azure,gcs]
# in this setup, you can detach from the process
uv run cheap-ocr modal --source "s3://in" --target "s3://out" --detach

Use on bare metal GPU

Follow these instructions, to run cheap-ocr on any GPU machine.

Clone the repository.

git clone https://github.com/blue-guardrails/cheap-ocr.git
cd cheap-ocr

Now build the docker image to run it or directly install the project into a virtual environment.

# docker
docker build -t cheap-ocr .

# Local in/out: mount the folders and reuse the cached weights.
docker run --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -v "$PWD/input:/data/in" -v "$PWD/output:/data/out" \
  cheap-ocr --source /data/in --target /data/out

# Cloud in/out: no mounts — pass URIs plus credentials as env (-e).
docker run --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY \
  cheap-ocr --source s3://bucket/in --target s3://bucket/out --limit 100


# Virtual environment (uv creates .venv automatically if needed)
uv sync --python 3.12 --frozen --no-dev --extra gpu --extra trt --extra s3   # add --extra gcs/azure as needed

uv run cheap-ocr --source ./input --target ./output
uv run cheap-ocr --source s3://bucket/in --target s3://bucket/out --limit 100

Use in code

from cheap_ocr import EngineConfig, OcrConfig, DocumentEngine

engine = DocumentEngine().start()          # loads layout model, boots vLLM

# Startup knobs via EngineConfig, per-run knobs via OcrConfig (or per call);
# unset fields fall back to CHEAP_OCR_* env vars, then the built-in defaults.
engine = DocumentEngine(
    EngineConfig(gpu_memory_utilization=0.7),
    run_config=OcrConfig(pdf_dpi=150),
).start()

result = await engine.ocr(pdf_bytes)            # one PDF, in memory
print(result.markdown, result.tokens.total_tokens)

async for doc in engine.stream(documents, config):   # many, as each finishes
    ...

summary = await engine.run("s3://in", "s3://out")    # storage-to-storage, resumable

Outputs

For each foo.pdf under the source, cheap-ocr writes one directory named after the relative stem in the target:

• foo/content.md — GLM-OCR-formatted Markdown
• foo/layout.json — per-page region JSON
• foo/stats.json — per-document timings, token usage, region counts and completion marker

A run-level _run_summary.json is written too, with document/page/token totals, throughput, and per-batch status.

Configuration

Most settings resolve the same way: explicit value → CHEAP_OCR_* environment variable → built-in default. "Explicit" means a CLI flag (--pdf-dpi 150, --max-num-seqs 512, --gpu H100) or a constructor argument in code (DocumentEngine(EngineConfig(...)), OcrConfig(...)). Plain CLI routing flags are called out below. OCR crops are always sent to vLLM as inline base64 data URLs.

The tables below list every operational flag on cheap-ocr and cheap-ocr modal (except --help, which prints the command help and exits).

Common CLI flags (both commands):

Flag	Default	Explanation
--source	./input	Local folder/file or cloud URI containing PDFs. Plain CLI flag.
--target	./output	Local folder or cloud URI for OCR outputs. Plain CLI flag.
--limit	all incomplete PDFs	Maximum number of incomplete PDFs to process in this run. Plain CLI flag.
--log-level	INFO	Log level (DEBUG, INFO, WARNING, ...); also reads CHEAP_OCR_LOG_LEVEL.

Run settings (OcrConfig, both commands):

Flag	Default	Explanation
--force/--no-force	off	Reprocess documents whose outputs are already complete.
--pdf-dpi	100	PDF render DPI. 100 is throughput-tuned; raise for accuracy.
--pdf-max-side	3500	Longest rendered page side in pixels; larger pages are downscaled.
--page-chunk-size	8	Pages streamed through render → layout → crop → OCR as one unit.
--jpeg-quality	90	JPEG quality for OCR crops before base64 encoding.
--max-tokens-text	2048	Maximum generated tokens for text regions.
--max-tokens-formula	2048	Maximum generated tokens for formula regions.
--max-tokens-table	4096	Maximum generated tokens for table regions.
--temperature	0.0	OCR generation temperature.
--top-p	0.00001	OCR generation nucleus-sampling top_p.
--max-inflight-pdfs	16 local, 32 on Modal	Documents rendered, laid out, and cropped concurrently. Modal uses 32 only when neither flag nor env var is set.
--ocr-request-concurrency	1024	Concurrent OCR requests to the vLLM server; keep near --max-num-seqs.
--storage-check-workers	64	Threads reading completion markers when resuming against cloud targets.

Engine settings (EngineConfig, both commands; startup-scoped):

Flag	Default	Explanation
--vllm-port	8000	Local port for the OpenAI-compatible vLLM server.
--vllm-health-timeout	900	Seconds to wait for the vLLM server to become healthy.
--gpu-memory-utilization	0.60	vLLM's share of GPU memory; remaining headroom is for the co-located layout model.
--max-model-len	8192	vLLM maximum model context length.
--max-num-seqs	1024	vLLM maximum concurrent sequences. OOM-safe to raise within the reserved KV pool.
--max-num-batched-tokens	16384	vLLM batched-token budget.
--api-server-count	4	vLLM API server processes; scale with available CPU cores.
--speculative-config	{"method": "mtp", "num_speculative_tokens": 3}	JSON passed to vLLM --speculative-config; an empty string disables it.
--vllm-extra-args	empty	Extra shell-style arguments appended to vllm serve.
--layout-backend	onnx	Layout detector backend: onnx (ONNX Runtime + TensorRT FP16) or transformers (PyTorch).
--layout-batch-size	4	Layout detection batch size; small batches interleave best with OCR.
--layout-threshold	backend default	Layout detection score threshold: 0.5 for onnx, 0.3 for transformers.
--trt-layout-cache	/root/.cache/vllm/trt_layout	TensorRT layout engine cache directory.
--trt-builder-opt-level	1	TensorRT builder optimization level.
--crop-encode-workers	8	Threads for crop/resize/JPEG/base64 preparation.

Batch policy (BatchPolicy, both commands):

Flag	Default	Explanation
--batch-docs	64	Maximum PDFs per GPU batch.
--batch-bytes-mb	512	Approximate maximum input size per GPU batch, in MB (CHEAP_OCR_BATCH_MAX_BYTES is bytes).
--batch-pages	unlimited	Maximum PDF pages per GPU batch.
--batch-retries	3	Retries for a failed or preempted GPU batch.
--batch-retry-backoff-seconds	10	Initial retry backoff seconds for failed batches.

Modal-only flags (cheap-ocr modal):

Flag	Default	Explanation
--detach/--no-detach	off	Submit a detached cloud run and return immediately. Requires cloud source and cloud target. Plain CLI flag.
--app-name	cheap-ocr	Modal app name.
--gpu	A100-40GB	Modal GPU type.
--cpu-cores	16	Modal CPU request.
--cpu-limit	none	Optional Modal CPU limit.
--scaledown-window	900	Modal container scaledown window in seconds.
--startup-timeout	1200	Modal container startup timeout in seconds.
--timeout	21600	Modal call timeout in seconds.
--max-containers	1	Maximum Modal GPU containers.
--modal-secrets	none	Comma-separated Modal secret names mounted into workers (CHEAP_OCR_MODAL_SECRET_NAMES).

Development

Install the development environment with uv:

uv sync --frozen --group dev

Common checks are available through the Makefile:

make lint          # Ruff lint checks
make format-check  # verify Ruff formatting
make types         # Pyright, including the Modal extra
make unit          # GPU-free test suite
make check         # lint, formatting, types, and tests

GitHub Actions workflows:

• .github/workflows/ci.yml runs lint, formatting, type checks, and tests on pull requests and pushes to main. Tests run on Python 3.12, 3.13, and 3.14.
• .github/workflows/zizmor.yml runs zizmor against the GitHub Actions workflows and uploads results to GitHub code scanning.
• .github/workflows/pypi_release.yml builds and publishes releases to PyPI using trusted publishing when a v* tag is pushed.

All workflow actions are pinned to full commit SHAs. When updating actions, use the newest available action version and update both the SHA and the version comment.

Release flow

Releases are published from tags through PyPI trusted publishing; no PyPI token is stored in GitHub.

One-time setup:

1. Create a GitHub environment named pypi.
2. Configure a PyPI trusted publisher for this repository with workflow .github/workflows/pypi_release.yml and environment pypi.

To publish a release:

1. Ensure main is green.

2. Choose the version, e.g. 1.2.3.

3. Create and push a matching tag:

   git tag v1.2.3
   git push origin v1.2.3
   

The release workflow strips the leading v, updates the package version for the build, builds the wheel and source distribution, smoke-tests both artifacts, and publishes them with uv publish --trusted-publishing always.

Acknowledgements

This work is only a thin layer on strong foundations:

• GLM OCR SDK
• PaddlePaddle/PP-DocLayoutV3
• bndos/pp-doclayout-v3-trt

License

Apache-2.0.