Skip to main content

LLM Sniffer

LLM Sniffer is an OpenAI-compatible reverse proxy with request/response inspector.

Features

  • Reverse Proxy: OpenAI-compatible API proxy
  • Request/Response Inspector: Monitor and inspect all LLM requests and responses
  • SSE Support: Server-Sent Events for streaming responses
  • Modern UI: Clean web interface for inspecting traffic
  • Image Preview: Display images (URL and base64) in messages with thumbnail and fullscreen view
  • Multi-Select & Export: Select multiple records in the sidebar and export as JSON
  • Runtime Settings: Adjust max record count (up to 300) directly from the WebUI
  • Multi-Upstream Support: Configure multiple LLM backends
  • Dynamic Configuration: Switch between upstreams via command line or config
  • Trace Save: Save request/response JSON to files for debugging and analysis

Installation

pip install llm-sniffer

Quick Start

Start the proxy server with default settings:

llm-sniffer

Configure your LLM client to use:

http://127.0.0.1:7654/v1

Then open http://127.0.0.1:7655 in your browser to inspect requests.

Command Line Options

Proxy Server Options

llm-sniffer [OPTIONS]

Options:
  --upstream-url URL     Direct upstream URL (highest priority)
  --upstream-name NAME   Use upstream from configuration file
  --upstream NAME/URL    Upstream name or URL (deprecated)
  --proxy-port PORT      Proxy service port (default: 7654)
  --ui-port PORT         UI service port (default: 7655)
  --max-records N        Maximum number of records to keep (default: 200)
  --think on|off         Enable/disable thinking mode (default: on)
  --host ADDRESS         Bind address (default: 127.0.0.1)
  --params JSON          Parameters to inject into each request body
  --save-trace           Enable saving request/response traces to files
  --output-dir DIR       Directory to save trace files (default: llm_sniffer_trace)

Configuration Management

# Initialize default config file
llm-sniffer config init

# List all configured upstreams
llm-sniffer config list

# Set active upstream
llm-sniffer config set kimi

# Add new upstream
llm-sniffer config add myserver --url http://localhost:8080 --description "My Server"

# Remove upstream
llm-sniffer config remove myserver

# Print config file path
llm-sniffer config path

Configuration File

Default config path: ~/.llm_sniffer/config.yaml

upstreams:
  local:
    url: http://127.0.0.1:8000
    api_key: ""
    description: Local LLM server (vLLM, Ollama, etc.)
  openai:
    url: https://api.openai.com
    api_key: ""
    description: OpenAI API
  qwen:
    url: https://dashscope.aliyuncs.com/compatible-mode
    api_key: ""
    description: Qwen (Alibaba Cloud)
  kimi:
    url: https://api.moonshot.cn
    api_key: ""
    description: Kimi (Moonshot AI)

active_upstream: local
proxy_port: 7654
ui_port: 7655
max_records: 200
think: on
host: 127.0.0.1
save_trace: false
output_dir: llm_sniffer_trace

WebUI Features

The built-in WebUI (default http://127.0.0.1:7655) provides a rich inspection experience:

Image Preview

Messages containing images (OpenAI image_url or Anthropic image content blocks) are rendered as clickable thumbnails. Click any thumbnail to view the full-size image in a modal overlay. Supports both URL-based and base64-encoded images.

Multi-Select & Export

  • Use checkboxes in the sidebar to select one or more records
  • Click 全选 to select/deselect all records
  • Click 导出 to download selected records as a JSON file

Runtime Settings

  • Click 设置 in the sidebar toolbar to adjust the maximum number of records kept in memory (1–300)
  • Changes take effect immediately without restarting the server

Upstream Selection Priority

  1. --upstream-url (command line) - Highest priority
  2. --upstream-name (command line)
  3. --upstream (command line) - Deprecated
  4. active_upstream in config file - Lowest priority

Trace Save

Enable trace save to save request/response JSON files for debugging and analysis.

Usage

# Enable trace save with default output directory
llm-sniffer --save-trace

# Enable trace save with custom output directory
llm-sniffer --save-trace --output-dir /path/to/traces

Output Structure

When trace save is enabled, files are saved in the following structure:

llm_sniffer_trace/
└── 20240101_120000_abc12345/
    ├── 0001_8f3d2a1b/
    │   ├── request.json
    │   └── response.json
    ├── 0002_9e4c3b2c/
    │   ├── request.json
    │   └── response.json
    └── ...
  • Each session gets its own directory with timestamp and session ID
  • Each request gets a numbered subdirectory
  • request.json: The request body sent to the upstream
  • response.json: The response body received from the upstream (merged for streaming)

Configuration

You can also configure trace save in the config file:

save_trace: true
output_dir: /my/custom/trace/path

Examples

# Use specific upstream URL
llm-sniffer --upstream-url http://localhost:8080

# Use upstream from config
llm-sniffer --upstream-name kimi

# Custom ports
llm-sniffer --proxy-port 8080 --ui-port 8081

# Start with custom upstream and inject parameters
llm-sniffer --upstream-name openai --params '{"temperature": 0.7}'

# Enable trace save
llm-sniffer --save-trace

# Enable trace save with custom output directory
llm-sniffer --save-trace --output-dir ./traces

License

MIT License

Metadata

Release files for llm-sniffer 0.8.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for llm-sniffer 0.8.2
File Size Uploaded Provenance Actions
llm_sniffer-0.8.2.tar.gz 57.8 kB Without provenance Details for llm_sniffer-0.8.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for llm-sniffer 0.8.2
File Interpreter ABI Platform Actions
llm_sniffer-0.8.2-py3-none-any.whl Python 3 none any Details for llm_sniffer-0.8.2-py3-none-any.whl

Total release size: 110.6 kB

Release files / llm_sniffer-0.8.2.tar.gz

Download URL llm_sniffer-0.8.2.tar.gz
Size 57.8 kB
Tags Source
SHA-256 checksum
How to use checksums
5f598c246378feed0e947dac50744b2ba7de731ea0a3bc5ecaaf148e3ad83bb4
BLAKE2b-256 checksum
How to use checksums
3eadf53c48a058d1d2414acc810c85fa9d2eaba490ce61efee16049e6345e8c7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.14

Release files / llm_sniffer-0.8.2-py3-none-any.whl

Download URL llm_sniffer-0.8.2-py3-none-any.whl
Size 52.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
320a060c36e090dc394baee309ad76c43e29656f3978a6a73584aee72da7214b
BLAKE2b-256 checksum
How to use checksums
8b5859b28762c97f60ccfb01330fc54ab7df7a443298719807a0c4b79f56605b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.14

Security for llm-sniffer 0.8.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page