e2eqavn is end to end pipeline for question answering
Project description
1. Introduction
-
e2eqavn is a 'simple' and 'easy to use' library that provides end to end pipeline for Question answering task.
-
e2eqavn build on Pytorch and provide command lined interface that allow user to train, test, and evaluate pipline.
2. Installation
pip install -U e2eqavn
3. Command line interface
Command | Function |
---|---|
e2eqavn train | - Training a model |
e2eqavn evaluate | - Evaluate the performance of a model |
e2eqavn test | - Performs the inference of a text input |
3.1 Training model
e2eqavn train --config [path_config_file]
This command starts a training pipeline that consists of retrieval or machine reading comprehension or both. If you want to train, you must provide path config yaml file. In this file config, you must setup the mode for training.
Example config file :
retrieval:
is_train: true # turn on mode training retrieval
.....
reader:
is_train: true # turn on mode training reader
You can see more detail parameter config in this link
- Arguments:
Required arguments:
--config CONFIG_FILE, -c CONFIG_FILE Path to the config file(architecture model, hyperparameter,..)
Optional arguments:
--help, -h Show this help message and exit.
- Example training CLI
e2eqavn train --config config.yaml
3.2 Evaluate pipeline
e2eqavn evaluate [MODE] --config [path_config_file]
--top_k_bm25 [INT_VALUE] # Optional, default value 10
--top_k_sbert [INT_VALUE] # Optional, default value 3
MODE must in:
- retrieval: Evaluate retrieval
- reader: Evaluate reader
- pipeline: Evalaute pipeline(retrieval + reader)
This command enable user to evaluate model in 3 ways:
- Retrieval model: Calculate recall@k, precision@k, ndcg metric and log result to csv file
- Reader model: Calculate 2 metrics: Exact match, F1 for machine reading comprehension task
- Pipeline model: Evaluate performance pipeline in Exact match and F1 score
Arguments:
Required arguments:
MODE Selection mode for evalaute(retrieval, reader, pipeline)
--config CONFIG_FILE Path to the config file.
--top_k_bm25 TOP_K_BM25 Top k document when retreival by BM25 algorithm
--top_k_sbert TOP_K_SBERT Top k document when retrieval by SentenceTransformer Algorithm
--logging_result_pipeline LOGGING_RESULT Logging result predict to json file when mode equal pipeline
Optional arguments:
--help, -h Show this help message and exit.
3.2 Testing
This command enable user for testing example with pipeline exist in local
e2eqavn evaluate [MODE] --config [path_config_file]
--question [STRING_QUESTION]
--top_k_bm25 [INT_VALUE] # Optional, default value 10
--top_k_sbert [INT_VALUE] # Optional, default value 3
--top_k_qa [INT_VALUE] # Optional, default value 1
MODE must in:
- retrieval: Evaluate retrieval
- reader: Evaluate reader
- pipeline: Evalaute pipeline(retrieval + reader)
Arguments:
Required arguments:
MODE Selection mode for evalaute(retrieval, reader, pipeline)
--config CONFIG_FILE Path to the config file.
--question QUESTION Which question do you want to ask?
--top_k_bm25 TOP_K_BM25 Top k document when retreival by BM25 algorithm
--top_k_sbert TOP_K_SBERT Top k document when retrieval by SentenceTransformer Algorithm
--top_k_qa TOP_K_QA Top k mrc result
Optional arguments:
--help, -h Show this help message and exit.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for e2eqavn-0.1.9-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 812dda82fc5cb1b0e1731918404267a2c6d62e0e12dc792ff64f55b7597901dc |
|
MD5 | c99327c0ee0cbba466e5bd3852d0bd6d |
|
BLAKE2b-256 | 7f1018537915a9fe29a8c66d0591a75e4dc776ad27b7f90ce06a2bb738e3318b |