This package is used for inducing longformer self attention over base pegasus abstractive summarization model to increase the token limit and performance.The Pegasus is a large Transformer-based encoder-decoder model with a new pre-training objective which is adapted to abstractive summarization. More specifically, the pre-training objective, called “Gap Sentence Generation (GSG)”, consists of masking important sentences from a document and generating these gap-sentences.On the other hand, the Longformer is a Transformer which replaces the full-attention mechanism (quadratic dependency) with a novel attention mechanism which scale linearly with the input sequence length. Consequently, Longformer can process sequences up to 4,096 tokens long (8 times longer than BERT which is limited to 512 tokens).This package plugs Longformers attention mechanism to Pegasus in order to perform abstractive summarization on long documents. The base modules are built on Tensorflow platform.
1 maintainer
Metadata
1 maintainer
Release files for LongPegasus 0.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| LongPegasus-0.3.tar.gz | 3.7 kB | Details |
Release files / LongPegasus-0.3.tar.gz
| Download URL | LongPegasus-0.3.tar.gz |
|---|---|
| Size | 3.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
89724595186b36de9989fe2307db0c12b7fa9a3a761030817b205bd9722f6f7f
|
|
BLAKE2b-256 checksum How to use checksums |
41be0b1c00041a52a3d9d8748b453587033c4d2a5ccec79a2e6bf3cce9b39381
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.5.0.1 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.8.3
|
