tfy-assistant-framework 0.1.0rc1

Framework for building AI assistants with streaming updates and user interactions

Tfy Assistant Framework

Overview

The Tfy Assistant Framework is a framework for building AI assistants with streaming updates and user interactions.

Installation

pip install tfy-assistant-framework

Usage

Simple chat assistant:

from tfy_assistant_framework.swarm import Agent, client

class Chat(Agent[Any]): ...

chat_agent = Chat(
    instructions="Chat with the user",
)

async def run_chat_assistant() -> None:
    async for agent_run in client.run(agent=chat_agent, messages=[], interactive=True):
        agent_run.debug_log()

This can be run in console directly to chat with the assistant. To serve the assistant via FastAPI,

js: JetStreamContext
assistant_serve = AssistantServe()

@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
    global js
    async with nats_connection() as (nc, js_):
        js = js_
        async with assistant_serve.init(nc=nc, app=app):
            yield

app = FastAPI(
    title="LLM Assistants Server",
    lifespan=lifespan,
    docs_url="/",
)

@app.post("/assistants/chat/task")
async def chat_task() -> CreateAssistantTaskRunResult:
    task_id = uuid.uuid4().hex
    return await assistant_serve.create_assistant_task(
        task_id=task_id,
        assistant_awaitable=run_chat_assistant(),
        assistant_update_sink=NATSAssistantUpdateSink(task_id=task_id, js=js),
    )

@app.post("/tasks/{task_id}/message")
async def send_message_to_assistant_task(
    task_id: str,
    message: PostAssistantTaskMessage,
) -> Response:
    return await assistant_serve.send_message_to_assistant_task(task_id, message)

For more examples, see the examples directory.