transformers-stream-generator
Description
This is a text generation method which returns a generator, streaming out each token in real-time during inference, based on Huggingface/Transformers.
Web Demo
- original
- stream
Installation
pip install transformers-stream-generator
Usage
- just add two lines of code before your original code
from transformers_stream_generator import init_stream_support
init_stream_support()
- add
do_stream=Trueinmodel.generatefunction and keepdo_sample=True, then you can get a generator
generator = model.generate(input_ids, do_stream=True, do_sample=True)
for token in generator:
word = tokenizer.decode(token)
print(word)
Example
1 maintainer
Release files for transformers-stream-generator 0.0.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| transformers-stream-generator-0.0.5.tar.gz | 13.0 kB | Details |
Release files / transformers-stream-generator-0.0.5.tar.gz
| Download URL | transformers-stream-generator-0.0.5.tar.gz |
|---|---|
| Size | 13.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
271deace0abf9c0f83b36db472c8ba61fdc7b04d1bf89d845644acac2795ed57
|
|
BLAKE2b-256 checksum How to use checksums |
42c265f13aec253100e1916e9bd7965fe17bde796ebabeb1265f45191ab4ddc0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.9.18
|
