Skip to main content

Pytorch implementation of pQRNN

Project description

banner

<center> Code style: black License: MIT </center>

Environment

Note: Because of recent pytorch change (>=1.7), it is not possible to run a QRNN layer without messing up the environment. See https://github.com/salesforce/pytorch-qrnn/issues/29 for details.

pip install -r requirements.txt

If you want to use a QRNN layer, please follow the instructions here to install python-qrnn first with downgraded torch <= 1.4.

Usage

Usage: run.py [OPTIONS]

Options:
  --task [yelp2|yelp5|toxic]      [default: yelp5]
  --b INTEGER                     [default: 128]
  --d INTEGER                     [default: 96]
  --num_layers INTEGER            [default: 2]
  --batch_size INTEGER            [default: 512]
  --dropout FLOAT                 [default: 0.5]
  --lr FLOAT                      [default: 0.001]
  --nhead INTEGER                 [default: 4]
  --rnn_type [LSTM|GRU|QRNN|Transformer]
                                  [default: GRU]
  --data_path TEXT
  --help                          Show this message and exit.

Datasets

  • yelp2(polarity): it will be downloaded w/ datasets(huggingface)
  • yelp5: json file should be downloaded to into data/
  • toxic: dataset should be downloaded and unzipped to into data/

Example: Yelp Polarity

python -W ignore run.py --task yelp2 --b 128 --d 64 --num_layers 4

Benchmarks(not optimized)

Model Model Size Yelp Polarity (error rate) Yelp-5 (accuracy) Civil Comments (mean auroc) Command
PQRNN (this repo) 78K 6.3 70.4 TODO --b 128 --d 64 --num_layers 4 --rnn_type QRNN
PRNN (this repo) 90K 5.5 70.7 95.57 --b 128 --d 64 --num_layers 1 --rnn_type GRU
PTransformer (this repo) 617K 10.8 68 86.5 --b 128 --d 64 --num_layers 1 --rnn_type Transformer --nhead 2
PRADO1 175K 65.9
BERT 335M 1.81 70.58 98.8562
  1. Paper
  2. Best Kaggle Submission

Credits

tensorflow

Powered by pytorch-lightning and grid.ai

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pytorch-pqrnn, version 0.1.0
Filename, size File type Python version Upload date Hashes
Filename, size pytorch_pqrnn-0.1.0-py3-none-any.whl (9.1 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size pytorch-pqrnn-0.1.0.tar.gz (10.2 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page