Skip to main content

A text generation plugin using local LLMs or other text generation methods. Builds on top of `transformers` by Hugging Face.

Project description

Nendo Plugin TextGen


nendo core


Website Twitter Discord


A text generation plugin using local LLMs or other text generation methods. Builds on top of transformers by Hugging Face.

Features

Requirements

Since we depend on transformers, please make sure that you fulfill their requirements. You also need Pytorch installed on your system, please refer to the pytorch installation instructions.

Installation

  1. Install Nendo
  2. pip install nendo-plugin-textgen

If you have a CUDA GPU you can install the following library for an additional speedup:

pip install flash-attn --no-build-isolation

Then set ATTN_IMPLEMENTATION=flash_attention_2 in your environment variables.

Usage

Take a look at a basic usage example below. For more detailed information, please refer to the documentation.

>>> from nendo import Nendo
>>> nd = Nendo(plugins=["nendo_plugin_textgen"])

>>> nd.plugins.textgen(prompt=["Tell me about your favorite song."])[0]

Contributing

Visit our docs to learn all about how to contribute to Nendo: Contributing

License

Nendo: MIT License

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nendo_plugin_textgen-0.1.0.tar.gz (7.4 kB view hashes)

Uploaded Source

Built Distribution

nendo_plugin_textgen-0.1.0-py3-none-any.whl (7.5 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page