fastseq 0.0.1

Efficient implementations of sequence models with fast performance

FastSeq

Introduction

FastSeq provides efficient implementations of the popular sequence models with fast performance for text generation, summarization, and translation tasks. It can automatically optimize the performance of the pupular NLP toolkits (e.g. FairSeq) by simply import fastseq.

Benchmark

• Run bart.large.cnn on NVIDIA-V100-16GB

BatchSize	32	64	128
FairSeq-0.9.0	4.2 samples/s	OOM	OOM
FairSeq-0.9.0 + FastSeq	9.5 samples/s	12.8 samples/s	13.9 samples/s

where:

• FairSeq-0.9.0 refers to the v0.9.0 branch of FairSeq

• FairSeq-0.9.0 + FastSeq runs FastSeq on top of FairSeq0.9.0

• Parameters: beam_size=4, lenpen=2.0, max_len_b=140, min_len=55, no_repeat_ngram_size=3

• More details can be found at tests/optimizer/fairseq/benchmark_fairseq_optimizer.py

Requirements and installation

• Python version >= 3.6
• torch >= 1.4.0
• fairseq >= 0.9.0

git clone https://github.com/microsoft/fastseq
cd fastseq
pip install --editable ./

Usage

Example

Only one line of code change is needed to use the optimizations provided by FastSeq.

# import fastseq at the beginning of your program
import fastseq
import torch

# Download BART already finetuned for MNLI
bart = torch.hub.load('pytorch/fairseq', 'bart.large.mnli')
bart.eval()  # disable dropout for evaluation

# Encode a pair of sentences and make a prediction
tokens = bart.encode('FastSeq optimizes FairSeq.', 'FastSeq accelerates FairSeq.')
bart.predict('mnli', tokens).argmax()  # 2: entailment

Run tests

# run a single test.
python tests/optimizer/fairseq/test_fairseq_optimizer.py

# run benchmark.
python tests/optimizer/fairseq/benchmark_fairseq_optimizer.py

# run all the tests.
python -m unittest discover -s tests/ -p '*.py'

Build

# build package
python setup.py sdist bdist_wheel

Code Style

Python coding style

Changes to Python code should conform to PEP 8. yapf can be used to help format the python code, and use pylint to check your Python changes.

# format the code by yapf
yapf --style pep8 -i -r PYTHON_FILE/PACKAGE

# run pylint check
pylint --rcfile=.pylintrc  PYTHON_FILE/PACKAGE

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [e-mail] with any additional questions or comments.