langdash 1.20.3

A simple library for interfacing with language models.

langdash

A simple library for interfacing with language models.

Currently in beta!

Features:

• Support for guided text generation, text classification (through prompting) and vector-based document searching.
• Lightweight, build-it-yourself-style prompt wrappers in pure Python, with no domain-specific language involved.
• Token healing and transformers/RNN state reuse for fast inference, like in Microsoft's guidance.
• First-class support for ggml backends.

Documentation: Read on readthedocs.io

Repository: main / Gitlab mirror

Installation

Use pip to install. By default, langdash does not come preinstalled with any additional modules. You will have to specify what you need like in the following command:

pip install --user langdash[embeddings,sentence_transformers]

List of modules:

• core:
  • embeddings: required for running searching through embeddings
• backends:
  • Generation backends: rwkv_cpp, llama_cpp, ctransformers, transformers
  • Embedding backends: sentence_transformers

Note: If running from source, initialize the git submodules in the langdash/extern folder to compile foreign backends.

Usage

Examples:

• Text generation
• Generating TV shows
• Embedding search

See examples folder for full examples.

Running the Examples

All examples can be ran with the following command:

python examples/instruct.py [model type] [model name or path]

You can specify additional model parameters using the -ae CLI argument, and passing a valid Python literal. For example, to run the chat example using the WizardLM model with context length of 4096, do:

python examples/chat.py llama_cpp /path/to/ggml-wizardlm.bin -ae n_ctx 4096

Some examples require you to specify the prompt format. Formats include: wizardlm (shortened Alpaca format without the first prompt line and # Instruction:), and alpaca (the full format). You will need to specify it for most of the examples:

python examples/instruct.py llama_cpp /path/to/ggml-wizardlm.bin -ae n_ctx 4096 --prompt-format wizardlm

For a full list, see the examples/_instruct_format.py file.

License

Apache 2.0