llm-sniffer 0.2.0

LLM Sniffer - OpenAI-compatible reverse proxy with request/response inspector

LLM Sniffer

LLM Sniffer is an OpenAI-compatible reverse proxy with request/response inspector.

Features

• Reverse Proxy: OpenAI-compatible API proxy
• Request/Response Inspector: Monitor and inspect all LLM requests and responses
• SSE Support: Server-Sent Events for streaming responses
• Modern UI: Clean web interface for inspecting traffic
• Multi-Upstream Support: Configure multiple LLM backends
• Dynamic Configuration: Switch between upstreams via command line or config

Installation

pip install llm-sniffer

Quick Start

Start the proxy server with default settings:

llm-sniffer

Configure your LLM client to use:

http://127.0.0.1:7654/v1

Then open http://127.0.0.1:7655 in your browser to inspect requests.

Command Line Options

Proxy Server Options

llm-sniffer [OPTIONS]

Options:
  --upstream-url URL     Direct upstream URL (highest priority)
  --upstream-name NAME   Use upstream from configuration file
  --upstream NAME/URL    Upstream name or URL (deprecated)
  --proxy-port PORT      Proxy service port (default: 7654)
  --ui-port PORT         UI service port (default: 7655)
  --max-records N        Maximum number of records to keep (default: 200)
  --think on|off         Enable/disable thinking mode (default: on)
  --host ADDRESS         Bind address (default: 127.0.0.1)
  --params JSON          Parameters to inject into each request body

Configuration Management

# Initialize default config file
llm-sniffer config init

# List all configured upstreams
llm-sniffer config list

# Set active upstream
llm-sniffer config set kimi

# Add new upstream
llm-sniffer config add myserver --url http://localhost:8080 --description "My Server"

# Remove upstream
llm-sniffer config remove myserver

# Print config file path
llm-sniffer config path

Configuration File

Default config path: ~/.llm_sniffer/config.yaml

upstreams:
  local:
    url: http://127.0.0.1:8000
    api_key: ""
    description: Local LLM server (vLLM, Ollama, etc.)
  openai:
    url: https://api.openai.com
    api_key: ""
    description: OpenAI API
  qwen:
    url: https://dashscope.aliyuncs.com/compatible-mode
    api_key: ""
    description: Qwen (Alibaba Cloud)
  kimi:
    url: https://api.moonshot.cn
    api_key: ""
    description: Kimi (Moonshot AI)

active_upstream: local
proxy_port: 7654
ui_port: 7655
max_records: 200
think: on
host: 127.0.0.1

Upstream Selection Priority

1. --upstream-url (command line) - Highest priority
2. --upstream-name (command line)
3. --upstream (command line) - Deprecated
4. active_upstream in config file - Lowest priority

Examples

# Use specific upstream URL
llm-sniffer --upstream-url http://localhost:8080

# Use upstream from config
llm-sniffer --upstream-name kimi

# Custom ports
llm-sniffer --proxy-port 8080 --ui-port 8081

# Start with custom upstream and inject parameters
llm-sniffer --upstream-name openai --params '{"temperature": 0.7}'

License

MIT License