The cat's meow in AI conversation management.
Project description
kittychat
The purrfect chat utils library! With Kittychat, you'll be feline fine as a kittycat, since it ensures your threads stay within the LLMs token limits by whisking away old messages.
install
pip install kittychat
usage
import logging
from kittychat.msg_dropping import MessageDropper
from kittychat.errors import NotEnoughTokens
thread = [
{
"content": "What is the weather like in Boston?",
"role": "user",
},
]
functions = [
{
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
},
"required": ["location"],
},
}
]
model = "openai/gpt-3.5-turbo-0613"
msgd = MessageDropper(model)
length, max_tokens, is_too_big = msgd.check_thread_length(thread, functions)
if is_too_big:
logging.warn(f"Thread too big: {length} tokens; max is {max_tokens} for model '{model}'.")
try:
cropped_thread = msgd.run(thread, functions)
except NotEnoughTokens as e:
raise YourPromptTooBigException(e.token_count, e.max_tokens, model)
return cropped_thread
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
kittychat-0.1.4.tar.gz
(3.7 kB
view hashes)
Built Distribution
Close
Hashes for kittychat-0.1.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4f56f3b3c9c2fdd28fe044bc0dc60b8220c842664613e936c2f06cf953b4cd8c |
|
MD5 | 8e2559276b951572f3d0b794e26ea9d9 |
|
BLAKE2b-256 | e8404ddbb6e012d30272b67b2da213603987de91b1dad399371a7e800145d3e2 |