Content ranking using ZDA model
Project description
Multi-Class Content Ranking using Seq-to-Seq Models
This repository contains code for a multi-class HTML content ranking model using BiLSTM-CRF, as described in the paper "A High-Speed Multi-Class Content Ranking using Seq-to-Seq Models".
Overview
The model classifies HTML content into 6 classes indicating importance:
- Title
- Main content
- Related parts
- Other topics
- Headers and footers
- Useless parts
The model uses only structural HTML features and is language agnostic.
Abstract
Content Ranking is an important task in many applications, such as documents ranking, text retrieval, text summarization and other natural language processing (NLP) procedures. Existing approaches define content ranking as a binary classification boilerplate removal, in which a text is either boilerplate or main-content. While boilerplate removal provides a basic mean for text ranking, in many applications, more importance levels are required to distinguish between different parts of an HTML document.
Based on a new dataset of segmented and labeled HTML documents, in this paper we introduce a high-speed multi-class sequence-to-sequence text classification model for determining the importance level of a text within six levels. As we use only HTML’s structural features in the training process, our model is language-agnostic and also it can be fine-tuned on a small dataset of web pages of other languages.
The involved feature extraction uses an optimized algorithm in terms of speed, which makes the end-to-end inference on an html page in about and less than 25ms, which is a huge benefit in terms of scalability of the model. Besides, our model achieves the 83 percent score in accuracy and 74 percent score in average 6 classes f1 score, which is a prominent result.
Dataset
The dataset used for this project can be found at ZDA Dataset. It contains 1602 labeled HTML documents across 6 classes:
- Title
- Main content
- Related parts
- Other topics
- Headers and footers
- Useless parts
Results
The model achieves 83% accuracy and 74% average F1 score on the test set.
Confusion Matrix
The confusion matrix shows the model confuses classes with adjacent importance levels:
Example Output
The model generalizes well to new HTML pages:
Usage
See the examples for scripts showing end-to-end usage.
Basic usage:
from content_ranker import predict
model = predict.load_model("path/to/model.pt")
html_string = load_html(...)
features = extract_features(html_string)
tags = model.predict(features)
Repository Structure
.
├── Dockerfile
├── examples/ - Example scripts showing usage
├── main.py - Main entry point
├── pyproject.toml - pypi config
├── README.md
├── requirements.txt - Python requirements
├── src/ - Source code
│ └── content_ranker/ - Package containing model code
│ ├── features/ - Feature extraction
│ ├── models/ - PyTorch model definitions
│ ├── resources/ - Resources like pretrained models
│ └── utils/ - Utility functions
└── tests/ - Tests
Citation
Please cite the paper if you use this repository:
@article{ashourisefat-raziei-2023-content-ranker,
title={A High-Speed Multi-Class Content Ranking using Seq-to-Seq Models},
author={AshouriSefat, Soroush and Kazemi, Reza and Alikhany, Morteza and Raziei, Mohammad and Amini, Arash},
journal={IEEE Access},
volume={9},
pages={XXXXX--XXXXX},
year={2023},
publisher={IEEE}
}
Contact
For any questions, feel free to open an issue or contact us at soroush_ashourisefat@ee.sharif.edu or mohammadraziei1375@gmail.com .
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for content_ranker-0.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eb478f2aba463c17ce4b9e51ce248278e6fbc36046213fd86718b3eb0032b3e9 |
|
MD5 | 47d6849c01d29f0dfd1d54d61aca4df7 |
|
BLAKE2b-256 | 27febc07a86d20d8e34b51f49c4655c38f1b889f15fde6e791f6f57b54d6137c |