superduper-llamacpp 0.0.5

Superduper allows users to work with self-hosted LLM models via [Llama.cpp](https://github.com/ggerganov/llama.cpp).

superduper_llamacpp

Superduper allows users to work with self-hosted LLM models via Llama.cpp.

Installation

pip install superduper_llamacpp

API

• Code
• API-docs

Class	Description
superduper_llamacpp.model.LlamaCpp	Llama.cpp connector.
superduper_llamacpp.model.LlamaCppEmbedding	Llama.cpp connector for embeddings.

Examples

LlamaCpp

from superduper_llama_cpp.model import LlamaCpp

model = LlamaCpp(
    identifier="llm",
    model_name_or_path="mistral-7b-instruct-v0.2.Q4_K_M.gguf",
)
model.predict("Hello world")