Nendo Plugin TextGen
A text generation plugin using local LLMs or other text generation methods.
Builds on top of transformers by Hugging Face.
Features
- Generate text from a prompt using a local LLM
- Access all LLMs from the Hugging Face Model Hub
Requirements
Since we depend on transformers, please make sure that you fulfill their requirements.
You also need Pytorch installed on your system, please refer to the pytorch installation instructions.
Installation
- Install Nendo
pip install nendo-plugin-textgen
If you have a CUDA GPU you can install the following library for an additional speedup:
pip install flash-attn --no-build-isolation
Then set ATTN_IMPLEMENTATION=flash_attention_2 in your environment variables.
Usage
Take a look at a basic usage example below. For more detailed information, please refer to the documentation.
>>> from nendo import Nendo
>>> nd = Nendo(plugins=["nendo_plugin_textgen"])
>>> nd.plugins.textgen(prompt=["Tell me about your favorite song."])[0]
Contributing
Visit our docs to learn all about how to contribute to Nendo: Contributing
License
Nendo: MIT License
1 maintainer
Release files for nendo-plugin-textgen 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| nendo_plugin_textgen-0.1.1.tar.gz | 7.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| nendo_plugin_textgen-0.1.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 15.0 kB
Release files / nendo_plugin_textgen-0.1.1.tar.gz
| Download URL | nendo_plugin_textgen-0.1.1.tar.gz |
|---|---|
| Size | 7.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
644a56daa44df1956a53caf186431c81b18ae2aabee15d4322e43491516b241e
|
|
BLAKE2b-256 checksum How to use checksums |
b3df719906ec71a373fb53234d54fe3b95d2dc0de9208e03430cd1dd7653fc92
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.7.1 CPython/3.9.16 Linux/6.6.20_1
|
Release files / nendo_plugin_textgen-0.1.1-py3-none-any.whl
| Download URL | nendo_plugin_textgen-0.1.1-py3-none-any.whl |
|---|---|
| Size | 7.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
1a6a0945a3b06fc09995caa89d25d7a90920bdc646a959ec05e5611391c5a7c1
|
|
BLAKE2b-256 checksum How to use checksums |
561aca80f02bb15dbcf2d33b06b03514d7d18cd8cf57edd562a2b1a6c0932a5a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.7.1 CPython/3.9.16 Linux/6.6.20_1
|
