table-transformer 1.0.2

Table Transformer

Table Transformer Library

Original repository: https://github.com/microsoft/table-transformer

Introduction

This is the Table Transformer Model developed by Brandon Smock et al. of Microsoft AI. This repository consists of two big models: Table Detection (DETR) and Table Structure Recognition (TATR) for detecting and extracting table infomation into popular formats such as CSV or HTML table.

Usage

You can initialize the pipeline by calling the function.

    table_extractor = TableExtractionPipeline(
        str_config_path=config_path,
        str_model_path=model_path,
        str_device=device,
        structure_class_thresholds=thresholds,
        onnx_str_model_path=onnx_model_path)

with the following parameters:

• str_config_path: path to the structure recognition config json file (you can find one in src/structure_config.json)
• str_model_path: path to the PyTorch model for table structure recogntion (you can download from here or find the pre-trained model in model/)
• str_device: inference device, either cuda or cpu.
• structure_class_thresholds: a dictionary of recognition thresholds, see get_structure_class_thresholds() in src/inference for more details.
• onnx_str_model_path: path to the ONNX Table Structure Recognition model

The full model usage can be found here:

from table_transformer import (TableExtractionPipeline,
                               get_structure_class_thresholds,
                               visualize_cells)
from PIL import Image
import os
from tqdm import tqdm

# Get the structure class thresholds
structure_class_threshold = get_structure_class_thresholds(
    table=0.5,
    table_column=0.5,
    table_row=0.5,
    table_column_header=0.5,
    table_projected_row_header=0.5,
    table_spanning_cell=0.5,
    no_object=10
    )

# Initialize the Pipeline with the thresholds
table_transformer = TableExtractionPipeline(
    structure_class_thresholds=structure_class_threshold,
    onnx_str_model_path = "models/TATR-v1.1-All-msft.onnx")

# Open the table image in PIL.Image format
directory = "images\/"

# Loop through all images in a directory
for filename in tqdm(os.listdir(directory), desc="Processing images"):
    img = Image.open(directory + filename)
    # Run the pipeline, page_image is required
    result = table_transformer(
        page_image=img, onnx_inference=True,
        out_cells=True,
        out_html=True,
        out_csv=True,
        out_objects=True)

    # Print the result
    #print(result["objects"][0])
    # print(result["cells"][0])
    #print(result["html"][0])
    # print(result["csv"][0])

    # Visualize the result as Matplotlib plot and return the figure
    visualize = visualize_cells(img=img, cells=result["cells"][0], show_plot=False)
    #print(type(visualize)) # PIL.Image.Image
print("Test successful!")

Evaluation

With structure recognition, the original author has evaluated the v1.0 model on PubTables-1M with great results. With other datasets such as PubTabNet, the score is quite good.

You can check out the score and run the evaluation with your own dataset in this link.

Version history

• v1.0.1: Initial version