This release is a pre-release and may not be stable for production use.
AI DIAL Python SDK
Overview
Framework to create applications and model adapters for AI DIAL.
Applications and model adapters implemented using this framework will be compatible with AI DIAL API that was designed based on Azure OpenAI API.
Environment Variables
| Variable | Default | Description |
|---|---|---|
| DIAL_SDK_HEADERS_TO_PROXY | `` | A comma-separated list of headers that should be proxied from incoming requests to outgoing requests to the DIAL API. By default, no headers are proxied. |
| DIAL_SDK_SSE_HEARTBEAT_INTERVAL | When set, the SDK inserts ping comments into streaming chat completion responses after the response has been idle for the specified number of seconds, helping prevent read timeouts when the DIAL application isn't responsive. | |
| PYDANTIC_V2 | False | When True and Pydantic V2 is installed, DIAL SDK classes for requests/responses will be based on Pydantic V2 BaseModel. Otherwise, they will be based on Pydantic V1 BaseModel. |
Logging-related environment variables (log level, console format, and trace/span correlation) are documented in docs/logging.md.
Usage
Install the library using pip:
pip install aidial-sdk
Echo application example
The echo application example replies to the user by repeating their last message:
# Save this as app.py
import uvicorn
from aidial_sdk import DIALApp
from aidial_sdk.chat_completion import ChatCompletion, Request, Response
# ChatCompletion is an abstract class for applications and model adapters
class EchoApplication(ChatCompletion):
async def chat_completion(self, request: Request, response: Response) -> None:
# Get last message (the newest) from the history
last_user_message = request.messages[-1]
# Generate response with a single choice
with response.create_single_choice() as choice:
# Fill the content of the response with the last user's content
choice.append_content(last_user_message.text())
# DIALApp extends FastAPI to provide a user-friendly interface for routing requests to your applications
app = DIALApp()
app.add_chat_completion("echo", EchoApplication())
# Run built app
if __name__ == "__main__":
uvicorn.run(app, port=5000)
Run
python3 app.py
Check
Send the next request:
curl http://127.0.0.1:5000/openai/deployments/echo/chat/completions \
-H "Content-Type: application/json" \
-H "Api-Key: DIAL_API_KEY" \
-d '{
"messages": [{"role": "user", "content": "Repeat me!"}]
}'
You will see the JSON response as:
{
"choices":[
{
"index": 0,
"finish_reason": "stop",
"message": {
"role": "assistant",
"content": "Repeat me!"
}
}
],
"usage": null,
"id": "d08cfda2-d7c8-476f-8b95-424195fcdafe",
"created": 1695298034,
"object": "chat.completion"
}
Endpoints
Every deployment is exposed under two base paths:
| Base path | Description |
|---|---|
/openai/deployments/{deployment_name} |
The DIAL API path. The deployment name is a part of the path. |
/openai/v1 |
The Azure OpenAI v1 API path. The deployment name comes from the request headers. |
The following endpoints are served under each of them:
POST <base path>/chat/completionsPOST <base path>/ratePOST <base path>/tokenizePOST <base path>/truncate_promptGET <base path>/configurationPOST <base path>/embeddings
request.deployment_id is resolved from the first of the following sources that is set:
- the
X-DIAL-OVERRIDE-NAMEheader, - the deployment name from the path, for the
/openai/deployments/{deployment_name}requests, - the
X-DIAL-DEPLOYMENT-IDheader, which DIAL Core sets when it calls the/openai/v1endpoints.
An /openai/v1 request with none of the headers set fails with 500 Internal Server Error.
Development
Development Environment
This project requires Python ≥3.11 and Poetry ≥2.1.1 for dependency management.
Setup
-
Install Poetry. See the official installation guide.
-
(Optional) Specify custom Python or Poetry executables in
.env.dev. This is useful if multiple versions are installed. By default,pythonandpoetryare used.POETRY_PYTHON=path-to-python-exe POETRY=path-to-poetry-exe
-
Create and activate the virtual environment:
make init_env source .venv/bin/activate
-
Install project dependencies (including linting, formatting, and test tools):
make install
IDE configuration
The recommended IDE is VSCode. Open the project in VSCode and install the recommended extensions. VS Code is configured to use the Ruff formatter.
Alternatively you can use PyCharm that has built-in Ruff support.
Lint
Run the linting before committing:
make lint
To auto-fix formatting issues run:
make format
Test
Run unit tests locally for available python versions:
make test
Run unit tests for the specific python version:
make test PYTHON=3.11
Git hooks
You may optionally install Git hooks that will automatically run the linting step on Git push. You only need to do it once for the given repository.
make install_git_hooks
Clean
To remove the virtual environment and build artifacts run:
make clean
Build
To build the package run:
make build
Publish
To publish the package to PyPI run:
make publish
1 maintainer
Release files for aidial-sdk 0.43.0.dev1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| aidial_sdk-0.43.0.dev1.tar.gz | 46.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| aidial_sdk-0.43.0.dev1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 108.9 kB
Release files / aidial_sdk-0.43.0.dev1.tar.gz
| Download URL | aidial_sdk-0.43.0.dev1.tar.gz |
|---|---|
| Size | 46.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
481f68f3a33341055e6a787d2efd8e8be5e65e89005a2cdba4fa4e21b398e384
|
|
BLAKE2b-256 checksum How to use checksums |
dc505d81ce959c8e582ff33757ba35d1ef291e9ba151ba3393a0cdaacaca7ae3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.2.1 CPython/3.10.21 Linux/6.17.0-1022-azure
|
Release files / aidial_sdk-0.43.0.dev1-py3-none-any.whl
| Download URL | aidial_sdk-0.43.0.dev1-py3-none-any.whl |
|---|---|
| Size | 62.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
89bcf838ac74eb037414443d9f84fd78f02d6bc9001f5928f5bb6e31ba46b04d
|
|
BLAKE2b-256 checksum How to use checksums |
357dfb6d62b7cf8e6ba5936f6091bd2c6829dc1a2dc25672643e6dac64867b6e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.2.1 CPython/3.10.21 Linux/6.17.0-1022-azure
|
