mag-parse

One parse() for every document. PDFs go to mag-pdf; everything else goes to mag-file-handler.

from mag_parse import parse

r = parse("invoice.pdf")                      # quality="fast" (default)
r = parse("report.pdf", quality="accurate")   # table structure
r = parse("deck.pptx")                        # quality does not apply
r = parse(pdf_bytes, filename="report.pdf")   # bytes work too

r.markdown        # always populated
r.text            # always populated
r.format          # "pdf" | "docx" | "xlsx" | "eml" | ...
r.engine          # which engine actually ran
r.ok, r.error, r.warnings

The facade itself has zero required dependencies. Both engines are extras, and that is not tidiness — see Install.

Routing

Decided by magic bytes, never the extension. A .docx renamed to .pdf does not reach the PDF engine, and a PDF with no extension does.

parse(file, quality=...)
        │
   sniff %PDF
        │
   ┌────┴─────┐
 PDF        not PDF
   │            │
mag-pdf   mag-file-handler
   │            │
   └────┬───────┘
        ▼
    one Result

quality is a PDF-only knob

For every other format quality is accepted and ignored, so a caller never has to branch on file type. Result.quality is "" there — the knob did not default, it does not exist.

"accurate" selects mag-pdf's profile="tables", leaving table_mode at its default. Those are two different fast/accurate axes: the profile picks the pipeline, table_mode picks TableFormer's weights. The heavier weights measured 27.6 s against 16.1 s for one additional table row, so they are not what "accurate" buys — the table pipeline is.

Markdown, and how honest it is

Both .markdown and .text are always populated, on every path including failure. They are not equally structured, and structured_markdown says which you got:

A docx table currently arrives as tab-separated fragments rather than a pipe table. That is mag-file-handler's plain-text output passed through verbatim, and the warning says so rather than letting it pass as structured markdown. Teaching the engines real markdown per format is the planned follow-up; the API does not change when it lands.

Errors

A bad document is a Result, never an exception — check .ok / .error. Only two things raise, and both are the caller's to fix:

Result.status is "ok", "empty" or "error". empty is a failure — an extraction that returns nothing while reporting success is the silent failure both engines guard against, and the facade does not launder it.

gRPC server

For running the facade as a sidecar. One RPC, because there is one parse().

pip install 'mag-parse[all,serve]'
mag-parse-serve --host 0.0.0.0 --port 50051

# reflection is on, so no .proto needed to poke it
grpcurl -plaintext localhost:50051 list

# `content` is bytes, so JSON carries it base64-encoded
grpcurl -plaintext -d "{\"content\":\"$(base64 -w0 invoice.pdf)\",\"quality\":\"accurate\",\"filename\":\"invoice.pdf\"}" localhost:50051 mag_parse.v1.Parser/Parse

grpcurl -plaintext localhost:50051 mag_parse.v1.Parser/GetStatus
grpc_health_probe -addr localhost:50051

The schema is v1/parse.proto, which ships inside the wheel so a client in another language does not have to chase a GitHub URL for the version it happens to have. ParseReply carries the whole Result: markdown, text, status, ok, error, format, engine, quality, structured_markdown, page_count, warnings, and extra as a google.protobuf.Struct. pages is absent, because those are engine-specific page objects with no serialisation contract.

Five behaviours worth knowing before you deploy it:

• A bad document is an OK response with ok = false. An encrypted PDF is a fact about the document, not an outage; only a bad server returns a non-OK status.
• A missing engine is not a broken server. Both engines are extras, so a [pdf]-only install fails office calls FAILED_PRECONDITION with the exact pip command while the health service still says SERVING. What does turn it NOT_SERVING is an engine that is installed and failed to warm - for accurate, a missing model does not raise, it silently degrades to plain text-layer markdown.
• accurate runs one document at a time (mag-pdf holds a process-scope lock; pypdfium2 is not thread-safe). Run one process and scale with replicas - a second server in the same container loads its own copy of the models for no extra throughput. A full lane returns RESOURCE_EXHAUSTED after MAGPARSE_QUEUE_TIMEOUT_SECONDS rather than holding the stream open.
• Raise your client's receive limit. The server accepts up to MAGPARSE_MAX_MESSAGE_BYTES and sends replies of any size, but a stock gRPC client still refuses anything over 4 MB - which a large document's markdown will exceed. Pass ("grpc.max_receive_message_length", -1) in the channel options.
• No vision. There is no way to hand an LLM client across a wire, so the server is offline by construction and images come back as an error Result.

Server-level knobs only, from the environment: MAGPARSE_FAST_CONCURRENCY (2), MAGPARSE_OFFICE_CONCURRENCY (4), MAGPARSE_MAX_MESSAGE_BYTES (200 MB), MAGPARSE_QUEUE_TIMEOUT_SECONDS (30), MAGPARSE_SHUTDOWN_GRACE_SECONDS (30). Engine knobs are deliberately absent from both the environment and the wire: parse() hides num_workers, device and table_mode, and exposing them would mean bypassing the facade.

Timing

Latency is reported in the logs, not on the wire, in the same shape the rest of the platform logs it - an event name, then key=value, with duration_ms spelled the way every Temporal activity spells it:

parse_complete name=invoice.pdf lane=accurate format=pdf engine=mag-pdf/0.3.0 (tables)
  status=ok ok=True pages=12 chars=48311 bytes=2203114 queue_ms=0.1 duration_ms=27604.3
parse_failed   name=broken.pdf lane=fast bytes=914 queue_ms=0.0 duration_ms=12.4 error=RuntimeError
warmup_complete lane=accurate ready=True duration_ms=4182.9

queue_ms is separate from duration_ms on purpose. Rolled together, a saturated sidecar reads as a slow engine - and the obvious response to that reading, tuning the engine, is the wrong one. A caller that wants its own number measures the round trip, which is the one that includes the transport.

Container

One image, all three routes, no network at runtime. Built from the repo root, because it installs all three packages from this tree rather than from PyPI:

docker build -f mag_parse/Dockerfile -t magureai/mag-parse:0.1.0 .
docker run --rm -p 50051:50051 magureai/mag-parse:0.1.0

  magparse:
    image: magureai/mag-parse:0.1.0
    command: ["mag-parse-serve", "--host", "0.0.0.0", "--port", "50051"]
    environment:
      MAGPARSE_FAST_CONCURRENCY: 2
      MAGPARSE_OFFICE_CONCURRENCY: 4
      LOG_LEVEL: INFO
      # Deliberately NOT set: HF_HUB_CACHE / HF_HOME. The image bakes both
      # (/opt/models/hub, offline). Overriding either points the loader at an
      # empty cache, and for the accurate route that failure is SILENT - the
      # document degrades to plain text-layer markdown and still reports ok.
    # torch + TableFormer + the ONNX YOLO session resident is ~1.5-2.5g, plus a
    # few hundred MB per concurrent document.
    mem_limit: 4g

Weights are baked and HF_HUB_OFFLINE=1, so docker run --network none works. The image ships a HEALTHCHECK that speaks grpc.health.v1 - the same question grpc_health_probe asks - so compose's condition: service_healthy works without adding a probe binary. linux/amd64 only: neither liteparse nor extractous publishes an ARM64 wheel, so an arm64 image would lose both the PDF and the office routes rather than one of them.

The build gates are the image's test suite. They run every route for real - the OCR canary, TableFormer predicting, extractous on Linux, and all three routes driven over an actual gRPC socket - so a broken image fails docker build instead of quietly serving worse markdown.

The stubs under src/mag_parse/v1/ are committed, because a user running pip install has no protoc. After editing the .proto, regenerate them from mag_parse/src:

pip install 'mag-parse[proto]'   # grpcio-tools, pinned EXACTLY
python -m grpc_tools.protoc -I . --python_out=. --pyi_out=. --grpc_python_out=. mag_parse/v1/parse.proto

Use that pin rather than whatever pip resolves. protoc stamps its gencode version into the output and protobuf's rule is one-directional - the runtime must be newer than the gencode - so a current grpcio-tools emits gencode 7.x that refuses to import on the protobuf<7 runtime this package supports.

Install

pip install 'mag-parse[pdf]'        # PDFs; one dependency, Tesseract bundled
pip install 'mag-parse[office]'     # docx, xlsx, pptx, eml, html, md, txt, csv
pip install 'mag-parse[all]'        # both of the above
pip install 'mag-parse[tables]'     # adds quality="accurate" (~2 GB: torch)
pip install 'mag-parse[all,serve]'  # + the gRPC server (`mag-parse-serve`)

serve carries no engine on purpose - it is composed with whichever of the above you need, so a PDF-only sidecar does not inherit the office stack's platform limits.

The engines are extras rather than dependencies because extractous (under mag-file-handler) publishes no Linux ARM64 wheel and declares requires-python <3.14, while liteparse (under mag-pdf) has neither limit. Making the office side required would impose both constraints on PDF-only users:

Python 3.10+. Apache-2.0.