A flexible LLM manager that shifts between multiple models
Project description
ThinkShift_LLM
ThinkShift_LLM is a flexible LLM manager that shifts between multiple language models to ensure robust and uninterrupted AI-powered conversations.
Features
- Manages multiple LiteLLM clients
- Shifts between models when errors occur
- Supports both streaming and non-streaming completions
- Detailed logging of all interactions
- Round-robin client selection
Installation
You can install tShift_LLM using pip:
pip install tshift-llm
Usage
Here's a quick example of how to use tShift_LLM:
from tshift_llm import tShift_LLM, LiteLLMClient
clients = [
LiteLLMClient("gpt-3.5-turbo", "your-openai-key"),
LiteLLMClient("claude-2", "your-anthropic-key"),
LiteLLMClient("command-nightly", "your-cohere-key")
]
tshift_llm = tShift_LLM(clients)
response = tshift_llm.completion(
messages=[{"role": "user", "content": "Hello, how are you?"}]
)
print(response.choices[0].message.content)
For more detailed usage instructions, please refer to the documentation.
License
This project is licensed under the MIT License - see the LICENSE file for details.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tshift_llm-0.1.3.tar.gz
(4.2 kB
view hashes)
Built Distribution
Close
Hashes for tshift_llm-0.1.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3941fcb7c6fccc505a2b7266034d70517a5704d14422660570faf889469570b4 |
|
MD5 | fcfee5be3cde1274110fa5aed1c81587 |
|
BLAKE2b-256 | 8f12ee0ab5c53e7149213ebfc82a6001bcaeca7b93b1425cb837041eecc664c9 |