mcQA : Multiple Choice Questions Answering
Answering multiple choice questions with Language Models.
Installation
With pip
pip install mcqa
From source
git clone https://github.com/mcqa-suite/mcqa.git
cd mcQA
pip install -e .
Getting started
Data preparation
To train a mcQA model, you need to create a csv file with n+2 columns, n being the number of choices for each question. The first column should be the context sentence, the n following columns should be the choices for that question and the last column is the selected answer.
Below is an example of a 3 choice question (taken from the CoS-E dataset) :
| Context sentence | Choice 1 | Choice 2 | Choice 3 | Label |
|---|---|---|---|---|
| People do what during their time off from work? | take trips | brow shorter | become hysterical | take trips |
If you have a trained mcQA model and want to infer on a dataset, it should have the same format as the train data, but the label column.
See example data preparation below:
from mcqa.data import MCQAData
mcqa_data = MCQAData(bert_model="bert-base-uncased",
lower_case=True,
max_seq_length=256)
train_dataset = mcqa_data.read(data_file='swagaf/data/train.csv', is_training=True)
test_dataset = mcqa_data.read(data_file='swagaf/data/test.csv', is_training=False)
Model training
from mcqa.models import Model
mdl = Model(bert_model="bert-base-uncased",
device="cuda")
mdl.fit(train_dataset,
train_batch_size=32,
num_train_epochs=20)
Prediction
preds = mdl.predict(test_dataset,
eval_batch_size=32)
Evaluation
from sklearn.metrics import accuracy_score
from mcqa.data import get_labels
print(accuracy_score(preds, get_labels(train_dataset)))
References
| Type | Title | Author | Year |
|---|---|---|---|
| 📰 Paper | Explain Yourself! Leveraging Language Models for Commonsense Reasoning | Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong and Richard Socher | ACL 2019 |
| 📰 Paper | SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference | Rowan Zellers, Yonatan Bisk, Roy Schwartz and Yejin Choi | 2018 |
Metadata
Release files for mcqa 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| mcqa-0.1.1.tar.gz | 15.6 kB | Without provenance | Details for mcqa-0.1.1.tar.gz |
Release files / mcqa-0.1.1.tar.gz
| Download URL | mcqa-0.1.1.tar.gz |
|---|---|
| Size | 15.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7317fcf33d232cf42f70a097d9847e4f77433329188bff7635491387dcf5af13
|
|
BLAKE2b-256 checksum How to use checksums |
54a792e02e08b640c6b4f339b249211688fc52b09ef32efb603527b766e4f0f2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/3.7.1
|
Security for mcqa 0.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.