LLM-Mediator 0.9.8

Just a simple mediator for different LLM models.

LLM Mediator

Just a simple mediator for different LLM models Will cache the response for the same input text during debug and save money for you.

Features

• Cache
• GPT-3.5
• GPT-3.5-16k
• GPT-4
• GPT-4-32k
• LLaMA2
• Falcon

Quick Usage

Install:

pip install llm_mediator
# Install llm_mediator from github
pip install git+https://github.com/zeuscsc/llm_mediator.git

Usage:

model_name="GPT-4-32k"
model=LLM(GPT)
model.model_class.set_model_name(model_name)
response=model.get_response(system,assistant,user)

Where system, assistant, user are the input text, and response is the output text.

Set Environment Variables

Unix:

export OPENAI_API_KEY=your openai key (Nessary for GPT)
export TECKY_API_KEY=your tecky key (Nessary for GPT)

Windows:

$ENV:OPENAI_API_KEY="your openai key" (Nessary for GPT)
$ENV:TECKY_API_KEY="your tecky key" (Nessary for GPT)

Python: Create a

from llm_mediator import gpt
gpt.OPENAI_API_KEY="your openai key" (Nessary for GPT)
gpt.TECKY_API_KEY = "your tecky key" (Nessary for GPT)