llm-proxy-gateway 0.0.3

Proxy Server that converts llm request format from one provider to another

LLM Proxy

Overview

llm-proxy-gateway is a small FastAPI service that accepts requests in different LLM API formats and converts them into the OpenAI Responses API format.

Right now, the project supports these endpoint styles:

• OpenAI Chat Completions API
• OpenAI Responses API
• Claude Messages API

The upstream OpenAI call is always made with streaming enabled. The proxy can return either a normal response or a streamed response based on the incoming request.

Installation

1. Make sure you have uv installed.
2. Create a .env file in the project root or export variables like startup.sh.
3. Add your configuration values.
4. Install dependencies.
5. Start the server.

Example .env file:

LLM_PROVIDER=OPENAI
API_SERVER_PORT=11434
OPENAI_API_KEY=your_openai_api_key_here
OPENAI_MODEL=gpt-4
OPENAI_BASE_URL=https://api.openai.com/v1

Install dependencies:

uv sync

Start the server:

./startup.sh

Or run directly:

uv run llm-proxy-gateway

Sample Request

Health check:

GET http://localhost:11434/health

OpenAI Chat Completions request:

POST http://localhost:11434/v1/chat/completions
Content-Type: application/json

{
  "model": "oca/gpt5",
  "stream": false,
  "messages": [
    {
      "role": "user",
      "content": "Hello, how are you?"
    }
  ]
}

OpenAI Responses request:

POST http://localhost:11434/v1/responses
Content-Type: application/json

{
  "model": "oca/gpt5",
  "input": "Reply with the word OK if you can read this.",
  "stream": false
}

Claude Messages request:

POST http://localhost:11434/v1/messages
Content-Type: application/json

{
  "model": "oca/gpt5",
  "max_tokens": 128,
  "messages": [
    {
      "role": "user",
      "content": "Hello Claude, reply with a short greeting."
    }
  ]
}

You can also use the ready-made examples in test.http.