polarkv 1.1.0rc0

PolarKV distributed KVCache Pool backend for LLM inference

PolarKV

Distributed KVCache Pool backend for LLM inference. Supports SGLang and vLLM.

Installation

pip install polarkv

Build From Source

Build the native SDKs from the repository root first, then assemble the wheel here:

cd ..
./build.sh client

cd python
./build_whl.sh
pip install dist/polarkv-*.whl

Usage

SGLang

--hicache-storage-backend dynamic \
--hicache-storage-backend-extra-config '{"backend_name": "polarkv", \
    "module_path": "polarkv.adapters.sglang", \
    "class_name": "PolarKVCache", \
    "polarkv_config_file":"/path/to/config.yaml", \
    "interface_v1":1 \
}'

vLLM

--kv-transfer-config '{"kv_connector": "PolarKVCache", \
    "kv_role": "kv_both", \
    "kv_connector_module_path": "polarkv.adapters.vllm", \
    "kv_connector_extra_config": {"config_file":"/path/to/config.yaml"} \
}'