faster-table-recognizer

Fast, dependency-light table detection for document images — a from-scratch Rust reimplementation of microsoft/table-transformer-detection (the DETR model that locates tables on a page), shipped as a pip install-able Python package.

• No PyTorch, no transformers, no ONNX. One ~1.8 MB native extension.
• Numerically identical output to the HuggingFace + PyTorch pipeline.
• Matches PyTorch/oneDNN's multithreaded speed on CPU, with a ~19× faster cold start and ~2× smaller memory footprint (see Benchmarks).
• Weights (~115 MB) are downloaded and cached automatically on first use.

pip install faster-table-recognizer

Linux x86-64 wheels are published to PyPI. The hot kernels are hand-written AVX2 selected at runtime, so the wheel runs on any x86-64 CPU (AVX2 is used when present). For other platforms, build from source.

---

Quickstart (Python)

from faster_table_recognizer import TableRecognizer

# First construction downloads the weights (~115 MB) to a local cache.
rec = TableRecognizer(threshold=0.9)

with open("page.png", "rb") as f:
    image_bytes = f.read()

# 1) Get bounding boxes (original-image pixel coordinates)
for det in rec.detect(image_bytes):
    print(det["label_name"], round(det["score"], 4), det["box"])
# -> table 0.9998 [202.02, 210.86, 1119.51, 385.44]

# 2) Get cropped table images (JPEG bytes), padded by 20 px like the reference service
for i, jpg in enumerate(rec.detect_tables(image_bytes)):
    with open(f"table_{i}.jpg", "wb") as out:
        out.write(jpg)

TableRecognizer loads the model once and is thread-safe — create a single instance and share it. Inference releases the GIL, so concurrent calls run in parallel.

API

TableRecognizer(model_path=None, threshold=0.9)

Construct a recognizer.

• model_path — path to a model.safetensors. If omitted, the stock microsoft/table-transformer-detection weights are downloaded from HuggingFace and cached (see Model cache).
• threshold — default confidence threshold (0–1) used when a method is called without its own threshold.

rec.detect(image_bytes, threshold=None) -> list[dict]

Detect tables. Returns one dict per detection:

{
  "score": 0.9998,              # confidence
  "label": 0,                   # 0 = table, 1 = table rotated
  "label_name": "table",
  "box": [x0, y0, x1, y1],      # xyxy, original-image pixels
}

image_bytes is the raw content of any common image (PNG, JPEG, BMP, TIFF, WebP).

rec.detect_tables(image_bytes, threshold=None, pad=20.0) -> list[bytes]

Detect and crop each table. Returns a list of JPEG-encoded crops (bytes). Each box is expanded by pad pixels (clamped to the image) before cropping.

rec.detect_tables_base64(image_base64, threshold=None, pad=20.0) -> list[str]

Same as detect_tables, but takes a base64-encoded image string and returns base64-encoded JPEG crops. This mirrors the /detect_tables HTTP contract used by the editor_api service, so it is a direct drop-in.

import base64
b64 = base64.b64encode(open("page.png", "rb").read()).decode()
crops = rec.detect_tables_base64(b64)   # list[str] of base64 JPEGs

Model cache

The weights are stored once per user and reused. The directory is chosen in this order:

1. $FTR_CACHE_DIR
2. $XDG_CACHE_HOME/faster-table-recognizer
3. ~/.cache/faster-table-recognizer

To use your own weights (or pre-seed the cache in an offline/CI environment), pass model_path= or drop a model.safetensors into the cache directory:

rec = TableRecognizer(model_path="/models/table-transformer-detection/model.safetensors")

# pre-seed the cache (no network at runtime)
mkdir -p ~/.cache/faster-table-recognizer
cp model.safetensors ~/.cache/faster-table-recognizer/

Threads

Inference parallelises across CPU cores with Rayon. Cap it with the standard Rayon env var if you run several workers:

RAYON_NUM_THREADS=4 python app.py

---

Benchmarks

AMD Ryzen 7 3700X (8 cores / 16 threads, AVX2, no AVX-512), single 1362×1760 page, steady-state medians, full pipeline (decode + preprocess + inference):

	PyTorch + transformers	faster-table-recognizer
Best multi-thread inference	~189 ms (8 threads)	~196 ms (all cores) — parity
As deployed (torch.set_num_threads(1))	~699 ms	~196 ms (3.5× faster)
Cold start (import + load model)	~5.5 s	~0.3 s (~19×)
Steady RSS (weights loaded)	761 MB	~246 MB (0.2: drops mmap + redundant f32 copies)
Install	torch + transformers + timm (~2–3 GB)	one ~1.8 MB wheel
Detection output	—	identical (max abs logit Δ ≈ 1e-5)

Starting from a 2× deficit, from-scratch AVX2 kernels (register-blocked + broadcast GEMMs) and blocked, cache-resident Winograd F(2,3) convolutions bring multithreaded throughput to parity with the fully-tuned oneDNN path. PyTorch remains faster single-threaded; this engine reaches parity by scaling cleanly across cores.

How it works

1. Preprocess — decode, resize so the longest side is 800 px (DETR DetrImageProcessor semantics, parallel triangle resampler), ImageNet-normalize.
2. Backbone — ResNet-18 with frozen BatchNorm folded into the conv weights; stride-1 3×3 convs run as blocked (cache-resident) Winograd F(2,3), the rest as im2col + AVX2 GEMM.
3. Transformer — sine 2D position embeddings, 6 pre-norm encoder layers, then 6 decoder layers with 15 learned object queries.
4. Heads — linear class head + 3-layer MLP box head.
5. Postprocess — threshold, cxcywh→xyxy in original pixels, pad, crop, JPEG.

Weights are read straight from the stock HuggingFace model.safetensors; there is no offline conversion step.

Command-line

The crate also builds a standalone ftr binary:

ftr detect page.png                 # JSON detections
ftr crop   page.png ./out           # write cropped table JPEGs + JSON
ftr serve                           # newline-delimited JSON on stdin/stdout
ftr validate tests/golden           # numerical check vs the PyTorch golden reference

Build from source

# Python wheel
pip install maturin
maturin build --release --features python --out dist
pip install dist/faster_table_recognizer-*.whl

# or the CLI / Rust library
cargo build --release            # -> target/release/ftr
RUSTFLAGS="-C target-cpu=native" cargo build --release   # machine-tuned

License

MIT. Model weights © Microsoft, released under the MIT License.