TF-IDF + LogReg baseline for text classification
Project description
Text Classification Baseline
Pipeline for fast building text classification baselines with TF-IDF + LogReg.
Usage
Instead of writing custom code for specific text classification task, you just need:
- install pipeline:
pip install text-classification-baseline
- run pipeline:
- either in terminal:
text-clf-train --path_to_config config.yaml
- or in python:
import text_clf
text_clf.train(path_to_config="config.yaml")
NOTE: more about config file here.
No data preparation is needed, only a csv file with two raw columns (with arbitrary names):
text
target
The target can be presented in any format, including text - not necessarily integers from 0 to n_classes-1.
Config
The user interface consists of two files:
- config.yaml - general configuration with sklearn TF-IDF and LogReg parameters
- hyperparams.py - sklearn GridSearchCV parameters
Change config.yaml and hyperparams.py to create the desired configuration and train text classification model with the following command:
- terminal:
text-clf-train --path_to_config config.yaml
- python:
import text_clf
text_clf.train(path_to_config="config.yaml")
Default config.yaml:
seed: 42
path_to_save_folder: models
# data
data:
train_data_path: data/train.csv
valid_data_path: data/valid.csv
sep: ','
text_column: text
target_column: target_name_short
# tf-idf
tf-idf:
lowercase: true
ngram_range: (1, 1)
max_df: 1.0
min_df: 1
# logreg
logreg:
penalty: l2
C: 1.0
class_weight: balanced
solver: saga
n_jobs: -1
# grid-search
grid-search:
do_grid_search: false
grid_search_params_path: hyperparams.py
NOTE: grid search is disabled by default, to use it set do_grid_search: true
.
NOTE: tf-idf
and logreg
are sklearn TfidfVectorizer and LogisticRegression parameters correspondingly, so you can parameterize instances of these classes however you want. The same logic applies to grid-search
which is sklearn GridSearchCV parametrized with hyperparams.py.
Output
After training the model, the pipeline will return the following files:
model.joblib
- sklearn pipeline with TF-IDF and LogReg stepstarget_names.json
- mapping from encoded target labels from 0 to n_classes-1 to it namesconfig.yaml
- config that was used to train the modelhyperparams.py
- grid-search parameters (if grid-search was used)logging.txt
- logging file
Requirements
Python >= 3.6
Citation
If you use text-classification-baseline in a scientific publication, we would appreciate references to the following BibTex entry:
@misc{dayyass2021textclf,
author = {El-Ayyass, Dani},
title = {Pipeline for training text classification baselines},
howpublished = {\url{https://github.com/dayyass/text-classification-baseline}},
year = {2021}
}
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for text-classification-baseline-0.1.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | c2390cdc739ea7ef58c10a4b7c5e273fff2b51d2a19af03225b2e44c0c5fd072 |
|
MD5 | 12c5c0b3469fd6d5b6decae7333c029b |
|
BLAKE2b-256 | 6ff456bd214663fd1663a166bfe2ed6004a601353f4c5285c6f563d7c6d84cff |
Hashes for text_classification_baseline-0.1.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f5f6d126cfcf7dcf7833b08f87f6d2492aae535340c3db24d1ebad4a038039ee |
|
MD5 | e9793d06ffb5f14a0d9460a9b07a2608 |
|
BLAKE2b-256 | f98c41e1cf0cc08a886fdf73344f389c63847b97f4822717ded678ead3ea28a0 |