A local proxy for LLMs, providing a unified interface for multiple models and support for identity based authentication.
Project description
local-llm-proxy
Proxy local LLM calls to Azure
Example config
port: 8095
providers:
azure-o4-mini-env:
endpoint: "https://{service}.cognitiveservices.azure.com/openai/deployments/o4-mini/chat/completions?api-version=2025-01-01-preview"
model: o4-mini
auth:
type: apikey
envKey: AZURE_OPENAI_API_KEY
azure-o4-mini-key:
endpoint: "https://{service}.cognitiveservices.azure.com/openai/deployments/o4-mini/chat/completions?api-version=2025-01-01-preview"
model: o4-mini
auth:
type: apikey
key: djjskskskkkk
azure-o4-mini-azcli:
endpoint: "https://{service}.cognitiveservices.azure.com/openai/deployments/o4-mini/chat/completions?api-version=2025-01-01-preview"
model: o4-mini
auth:
type: azcli
Dev environment setup
# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
# Install packages
make install
# Lint and type check
make check
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
prompt_passage-0.0.1.tar.gz
(10.0 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file prompt_passage-0.0.1.tar.gz.
File metadata
- Download URL: prompt_passage-0.0.1.tar.gz
- Upload date:
- Size: 10.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.12.10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
c16382a544defe293d9077f2469a1f6dc93789a15e31ce14b83cef21eea21d5c
|
|
| MD5 |
a3ee7d75a6187a2482bbe35d99d014b9
|
|
| BLAKE2b-256 |
d23fb8f1d93da9cf0afbbd8aed0f8ebd53ea26d0da42392bda054a183d812455
|
File details
Details for the file prompt_passage-0.0.1-py3-none-any.whl.
File metadata
- Download URL: prompt_passage-0.0.1-py3-none-any.whl
- Upload date:
- Size: 9.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.12.10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
9e2545c2d1a3f1ea9ea19efc28bb32d9fb7342008e5573371e42edc8aab58284
|
|
| MD5 |
08780bd6d658732fc6ddbc5d5424459d
|
|
| BLAKE2b-256 |
13e3e564f807a03b5fe99396b6005b6ba65a9191bee8c882961ebd1434c6253d
|
