xPyD-proxy

Lightweight Prefill-Decode disaggregated proxy for LLM serving.

xPyD-proxy routes inference requests between prefill and decode nodes, enabling disaggregated LLM serving with load balancing, health monitoring, and fault tolerance.

Key Features

  • disaggregated serving — separate prefill and decode nodes for optimal resource utilization
  • Multiple scheduling policies — round-robin, consistent hash, cache-aware, power-of-two
  • Resilience — circuit breaker, health monitoring, automatic failover
  • Multi-model routing — serve multiple models through a single proxy
  • OpenAI-compatible API — drop-in replacement for vLLM/OpenAI endpoints
  • YAML configuration — declarative topology and settings

Install

pip install xpyd-proxy

Or as part of the full xPyD toolkit:

pip install xpyd

Quick Start

# Start with YAML config
xpyd proxy --config proxy.yaml

# Or with CLI args
xpyd proxy --model my-model \
  --prefill 127.0.0.1:8001 \
  --decode 127.0.0.1:8002

Part of xPyD

Component Description
xpyd-proxy disaggregated proxy
xpyd-sim OpenAI-compatible inference simulator
xpyd-bench Benchmarking & planning tool

📖 Full Guide → | 💡 Examples → | 🏗️ Contributing →

License

Apache 2.0 — see LICENSE

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for hlin99 from gravatar.com hlin99

Requires

Python >=3.10

Provides Extra

dev

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

xpyd_proxy-1.4.0.tar.gz (57.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

xpyd_proxy-1.4.0-py3-none-any.whl (68.4 kB view details)

Uploaded Python 3

File details

Details for the file xpyd_proxy-1.4.0.tar.gz.

File metadata

  • Download URL: xpyd_proxy-1.4.0.tar.gz
  • Upload date:
  • Size: 57.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for xpyd_proxy-1.4.0.tar.gz
Algorithm Hash digest
SHA256 9ba76ae822b46e7001193a89fce59f56099c5ef2908337de3e69f186a834db0c
MD5 4b9f505d0580ebc94f93c38ba9c5933c
BLAKE2b-256 946176fe94c78730704bd0a0f1268bcdf0cfb44d73605a496237c45eb21f2218

See more details on using hashes here.

Provenance

The following attestation bundles were made for xpyd_proxy-1.4.0.tar.gz:

Publisher: release.yml on xPyD-hub/xPyD-proxy

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file xpyd_proxy-1.4.0-py3-none-any.whl.

File metadata

  • Download URL: xpyd_proxy-1.4.0-py3-none-any.whl
  • Upload date:
  • Size: 68.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for xpyd_proxy-1.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 2469bcfcbac57d4bb85cffb49cb5f02d074169ba5a1dec611dad04da64c34124
MD5 a92643641283a61f6579431e575c909e
BLAKE2b-256 86659dcadb5376d5dc4a3a410eefff86090aa89eb2e716b23641929e438b7588

See more details on using hashes here.

Provenance

The following attestation bundles were made for xpyd_proxy-1.4.0-py3-none-any.whl:

Publisher: release.yml on xPyD-hub/xPyD-proxy

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page