FastAPI Reverse Proxy

A robust, streaming-capable reverse proxy for FastAPI/Starlette with built-in Latency-Based Load Balancing and Active Health Monitoring.

Features

• Async: Async by default.
• Httpx Pool: Async HTTPX Pool for proxying.
• Streaming Ready: Handles SSE (Server-Sent Events) and large payloads (such as big files) while keeping RAM usage low.
• WebSocket Support: Seamless bidirectional tunneling with automated subprotocol negotiation.
• Unified Load Balancing: Standard Round-Robin or Smart routing using a single utility.
• Latency-Based Routing: Automatically routes traffic to the fastest healthy server (HEAD probe).
• Advanced Overrides: Granular control over headers, body, and HTTP methods.
• Smart Error Mapping: Automatically converts upstream connection failures into standard HTTP 502 (Bad Gateway) and 504 (Gateway Timeout) responses.
• Resilient Handshakes: Customizable open_timeout for WebSockets to prevent proxy hangs during backend connection attempts.
• Version Agnostic: Automatically handles websockets library version differences (12.0+ vs Legacy).

Quick Start

Use the lifespan handler as shown for an easy launch.

The simplest way to use the proxy is to use proxy_pass and/or proxy_pass_websocket on the endpoints.

from fastapi import FastAPI, Request, WebSocket
from contextlib import asynccontextmanager

from fastapi_reverse_proxy import Proxy, proxy_pass, proxy_pass_websocket

@asynccontextmanager
async def lifespan(app: FastAPI):
    async with Proxy(app):
        yield

app = FastAPI(lifespan=lifespan)

# catch-all route. recommended for a reverse proxy
@app.api_route("/{path:path}", methods=["GET","POST","PUT","DELETE"]) # don't forget to add the methods.
async def index(req: Request):
    """
    You always need to pass the "Request" object and to specify the host
    If you don't add a path, it will be the same as the original (/login --> http://127.0.0.1/login)
    """
    return await proxy_pass(req, "http://127.0.0.1:8080")

🛡️ Resilience & Error Handling

Error Handlingfastapi-reverse-proxy transforms upstream crashes into meaningful HTTPException responses (e.g., 502 Bad Gateway or 504 Gateway Timeout).

This allows you to implement custom failover logic, retry mechanisms, or specific error pages.

For a full implementation of a primary-to-backup failover system, see the example.

Advanced Examples:

Check examples for full examples, including:

• Websocket Proxy
• Socket.IO Proxy
• Error Handling & Failover (examples/error_handling_example.py)

Advanced Proxying

The proxy_pass function and LoadBalancer.proxy_pass provide deep customization for upstream requests:

Monitoring & Configuration

HealthChecker (The Loop Owner)

The proactive component. It owns an internal asyncio background task that monitors backends.

• Immediate Start: When you enter the async with block (or call start()), the checker performs an immediate check of all backends. This eliminates the "cold-start" window where backends are unknown.
• Configuration Modes:
  • Standard: HealthChecker(["http://a", "http://b"], ping_path="/health")
  • Personalized: Pass a list of dictionaries for per-host settings:

    checker = HealthChecker([
        {"host": "http://api-1", "pingpath": "/v1/status", "maxrequests": 100},
        {"host": "http://api-2", "pingpath": "/health"}
    ])
    

• Properties:
  • ping_path: Get or set the global health check path (default: "/").

LoadBalancer (The Decision Utility)

A normal Python object that makes routing decisions based on its source.

• Stateful: While it has no background loop, it does track state (request counts for rate-limiting and the last time it pulled data from the health checker).
• No Lifecycle Needed: It relies on the HealthChecker (or a static list) for data and doesn't need explicit start/stop calls.

WebSocket Refinements

The library implements "deferred negotiation" for WebSockets:

1. The proxy receives the client's supported subprotocols from scope.
2. It establishes an upstream connection first.
3. Once the upstream accepts a protocol, the proxy calls websocket.accept(subprotocol=...) back to the client.
4. This ensures the entire tunnel (Client <-> Proxy <-> Upstream) uses the same negotiated protocol.
5. Handshake Timeout: Supports a customizable timeout parameter (default 10.0s) to prevent hangs if the backend is unresponsive.
6. Error Handling: Raises fastapi.WebSocketException when the upstream connection fails or is rejected with proper WS codes such as 1008 or 1011
7. Debug: Disruptions from either side are logged on debug level.

Robustness & Safety

• Termination Safety: Resource cleanup (closing httpx clients and sockets) is triggered even on task cancellation (BaseException).
• Introspection-Based Compatibility: Uses inspect.signature to automatically detect version-specific parameters in the websockets library.
• RFC 7230 Compliant Header Handling: Hop-by-hop headers (Connection, Transfer-Encoding, TE, Trailers, Keep-Alive, Proxy-Authenticate, Proxy-Authorization) are stripped from both outbound requests and responses, per spec. WebSocket handshake headers (Sec-WebSocket-Key, Upgrade, etc.) from the client are never forwarded to the target, avoiding handshake collisions.
• Content-Length Safety: Always stripped from the outbound request and recalculated by httpx (or sent chunked when streaming), preventing LocalProtocolError when override_body differs in size from the original request.
• Content-Encoding Passthrough: Compressed upstream responses (gzip, br, etc.) are streamed back to the client raw via aiter_raw(), with Content-Encoding preserved — the client decompresses it itself, avoiding corrupted/garbled response bodies.

Running Behind a Reverse Proxy (Nginx/Apache)

By default, proxy_pass and proxy_pass_websocket forward the client's original headers as-is — they do not set or rewrite X-Real-IP, X-Forwarded-For, X-Forwarded-Proto, or X-Forwarded-Host. If this library sits behind Nginx or Apache (the common setup), your upstream server is responsible for setting those headers before the request reaches this proxy:

location / {
    proxy_pass http://your-fastapi-app;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Host $host;
}

For WebSocket routes, Nginx also needs explicit upgrade handling:

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

location /ws/ {
    proxy_pass http://your-fastapi-app;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

Without this configuration, X-Forwarded-* headers will be empty or missing by the time they reach your application.

Streaming & Response Buffering

X-Accel-Buffering: no is set automatically on every proxied response to disable Nginx's response buffering for streamed content (SSE, chunked transfers, large file downloads). If deploying behind Apache instead, disable buffering via your Apache config (mod_proxy directives) — there's no equivalent response header Apache recognizes.