vllm-hust-clm-lifecycle

CLM: Context Lifecycle Management perception plugin for vLLM-HUST.

Consumes the scheduler-level request lifecycle events exposed by the host (RequestFinished / RequestPreempted, contract vllm.request-lifecycle.v1) and reports session-level KV growth to an out-of-tree CLM controller, which decides when to compact a long conversation.

Bundle ID: org.vllm-hust.clm-lifecycle (Extension Bundle v1, schema 1.0)

What it does

• Lifecycle-sink component (contract vllm.request-lifecycle.v1, execution plane scheduler): registers a sink on the engine's default-off typed EventBus. When a request finishes, is preempted, or has its KV actually reclaimed, the sink attributes the footprint to a session (engine-side session_id, or a __default__ bucket) and reports it to a controller endpoint.
• Reports distinguish decisions from reclaims. kv_growth carries kv_reclaim_deferred, which is true when the host deferred the block release (a connector, or an in-flight step); kv_reclaimed is the event that means the blocks really returned to the pool. A controller should not treat a deferred kv_growth as free capacity.
• Default-off, zero-overhead when absent: the events only exist on hosts that carry the request-lifecycle hook; without them register() skips gracefully. Reports never block serving (failures are swallowed).

Install

pip install vllm-hust-clm-lifecycle

Enable

Requires a host that exposes vllm.v1.events (vllm-hust with the request-lifecycle hook). Enable via plugin loading:

# runtime registration (vllm.general_plugins)
export VLLM_PLUGINS=ascend,clm_lifecycle
# ECPA static discovery and activation intent
vllm-hust-ext extension inspect org.vllm-hust.clm-lifecycle
vllm-hust-ext extension check org.vllm-hust.clm-lifecycle

Environment

Parse failures on these variables fall back to defaults (empty values are tolerated).

Reports never block the scheduler

emit() runs on the engine's scheduler thread, so it performs no I/O: it aggregates in memory and hands the report to a bounded queue drained by a private daemon thread. A slow or unreachable controller therefore cannot stall scheduling, and the bounded queue means it cannot grow memory without limit.

Every counter is available via ClmLifecycleSink.stats_snapshot() so "non-blocking" is observable rather than asserted:

The test suite drives a real localhost HTTP server that stalls for seconds and asserts that emit() stays far below the sender timeout, that drops are counted, and that close() neither hangs nor loses reports silently.

Disable

unset VLLM_CLM_CONTROLLER VLLM_CLM_REPORT_BLOCKS VLLM_CLM_REPORT_INTERVAL
# or simply do not list clm_lifecycle in VLLM_PLUGINS
pip uninstall vllm-hust-clm-lifecycle

Development

pip install -e ".[test]"
python -m pytest tests/
ruff check .

tests/conftest.py stubs vllm.v1.events before the plugin is imported, so the test suite runs without a vLLM install.

License

Apache-2.0