Idiomatic way to build chatgpt apps using async generators in python
Project description
turbo-chat
Idiomatic way to build chatgpt apps using async generators in python
The ChatGPT API uses a new input format called chatml. In openai's python client, the format is used something like this:
messages = [
{"role": "system", "content": "Greet the user!"},
{"role": "user", "content": "Hello world!"},
]
The idea here is to incrementally build the messages using an async generator and then use that to generate completions. Async generators are incredibly versatile and simple abstraction for doing this kind of stuff. They can also be composed together very easily. See example below.
Installation
pip install turbo-chat
Example
from typing import AsyncGenerator
from turbo_chat import (
turbo,
System,
User,
Assistant,
GetUserInput,
Generate,
run,
)
# Get user
async def get_user(id):
return {"zodiac": "pisces"}
# Set user zodiac mixin
@turbo()
async def set_user_zodiac(context: dict):
user_id: int = context["user_id"]
user_data: dict = await get_user(user_id)
zodiac: str = user_data["zodiac"]
yield User(content=f"My zodiac sign is {context['zodiac']}")
# Horoscope app
@turbo()
async def horoscope(context: dict):
yield System(content="You are a fortune teller")
async for (output, _) in run(set_user_zodiac()):
yield output
# Prompt runner to ask for user input
input = yield GetUserInput(message="What do you want to know?")
# Yield the input
yield User(content=input)
# Generate (overriding the temperature)
value = yield Generate(settings={"temperature": 0.9})
# Let's run this
app: AsyncGenerator[Assistant | GetUserInput, str] = horoscope({"user_id": 1})
_input = None
while True:
result, done = await run(app, _input)
if isinstance(result, GetUserInput):
_input = raw_input(result.message)
continue
if isinstance(result, Assistant):
print(result.content)
if done:
break
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
turbo_chat-0.1.2.tar.gz
(5.8 kB
view hashes)
Built Distribution
Close
Hashes for turbo_chat-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bee14f2fc3008dfb9da2a36d349185ead846642b3e0c6429f81e1cf3615ea812 |
|
MD5 | 3dfbf4b2c78056f0a056a552827d8500 |
|
BLAKE2b-256 | 12583e55ad3c32492694bd1e81fd10e2a898b1ae04b21bb8f02c624f8baa8efb |