Streaming enabled Assistants API
Project description
streaming_assistants
The official OpenAI Assistants API does not yet support streaming (although this functionality has been hinted at since launch back in November). Streaming is critical for a large subset of genai use cases and there has been significant feedback that it's the major blocker for addption of the Assistants API from many. We decided that we (and our users) couldn't wait so we implemented streaming support in Astra Assistants API.
OpenAI has now added streaming support with streaming runs. This libriary will now mainly be used to streamline support for third partry llm providers. We will continue to support our old streaming messages approach for existing users.
How to use
Install streaming_assistants using your python package manager of choice:
poetry add streaming_assistants
import and patch your client:
from openai import OpenAI
from streaming_assistants import patch
client = patch(OpenAI())
Server
The astra-assistants server code is now open source (Apache2)! Check it out here https://github.com/datastax/astra-assistants-api
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for streaming_assistants-0.15.6.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0cd13f1edd6f4028a372b6f2bd52a50b0528232e246bbedde8a50fe3815cebe2 |
|
MD5 | 57573c166073dc9d8503f529cc9037ad |
|
BLAKE2b-256 | ee9d3f42480a3705d926822586db14af36649e3a1e415f8cd393f7a28b36da70 |
Hashes for streaming_assistants-0.15.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 745d1e67b436427da4b1a32e2066afb7c4e52c92b92cab93b638404fad70ec16 |
|
MD5 | 1de9e7e79620225a42227f21616cd10d |
|
BLAKE2b-256 | 626e4a4efa07370fd1a4a20cabf7aeee5c4a4aaab73bc290c0805d763d27a555 |