pytorch-pqrnn 0.1.0

Pytorch implementation of pQRNN

Environment

Note: Because of recent pytorch change (>=1.7), it is not possible to run a QRNN layer without messing up the environment. See https://github.com/salesforce/pytorch-qrnn/issues/29 for details.

pip install -r requirements.txt

If you want to use a QRNN layer, please follow the instructions here to install python-qrnn first with downgraded torch <= 1.4.

Usage

Usage: run.py [OPTIONS]

Options:
  --task [yelp2|yelp5|toxic]      [default: yelp5]
  --b INTEGER                     [default: 128]
  --d INTEGER                     [default: 96]
  --num_layers INTEGER            [default: 2]
  --batch_size INTEGER            [default: 512]
  --dropout FLOAT                 [default: 0.5]
  --lr FLOAT                      [default: 0.001]
  --nhead INTEGER                 [default: 4]
  --rnn_type [LSTM|GRU|QRNN|Transformer]
                                  [default: GRU]
  --data_path TEXT
  --help                          Show this message and exit.

Datasets

• yelp2(polarity): it will be downloaded w/ datasets(huggingface)
• yelp5: json file should be downloaded to into data/
• toxic: dataset should be downloaded and unzipped to into data/

Example: Yelp Polarity

python -W ignore run.py --task yelp2 --b 128 --d 64 --num_layers 4

Benchmarks(not optimized)

Model	Model Size	Yelp Polarity (error rate)	Yelp-5 (accuracy)	Civil Comments (mean auroc)	Command
PQRNN (this repo)	78K	6.3	70.4	TODO	--b 128 --d 64 --num_layers 4 --rnn_type QRNN
PRNN (this repo)	90K	5.5	70.7	95.57	--b 128 --d 64 --num_layers 1 --rnn_type GRU
PTransformer (this repo)	617K	10.8	68	86.5	--b 128 --d 64 --num_layers 1 --rnn_type Transformer --nhead 2
PRADO1	175K		65.9
BERT	335M	1.81	70.58	98.8562

1. Paper
2. Best Kaggle Submission

Credits

tensorflow

Powered by pytorch-lightning and grid.ai