Pre-release

This release is a pre-release and may not be stable for production use.

lmcache logo

A KV Cache Management Layer for Scalable LLM Inference

Blog | Documentation | Join Slack | Community Meeting | Roadmap

GitHub Repo stars PyPI PyPI - Downloads GitHub commit activity GitHub contributors Good first issues Slack Ask DeepWiki

⭐ If LMCache helps you serve LLMs faster and cheaper, give us a star — it helps more teams discover the project.

Updates

  • [2026/05] 🔥 Agentic workload benchmark on AMD MI300X (blog).
  • [2026/04] 🔥 LMCache's new multiprocess (MP) architecture release (blog).
  • [2026/03] LMCache at GTC 2026 (post).
  • [2026/01] LMCache multi-node P2P CPU memory sharing, from experimental feature to production (blog).
More
  • [2025/11] LMCache x CoreWeave accelerate efficient LLM inference for Cohere (blog).
  • [2025/10] LMCache joins the PyTorch Foundation and Tensormesh unveiled (blog, PyTorch).
  • [2025/09] NVIDIA Dynamo integrates LMCache, accelerating LLM inference (blog).
  • [2025/08] 🎉 LMCache hits 5,000+ GitHub stars (blog).
  • [2025/08] LMCache supports gpt-oss (20B/120B) on day 1 (blog).
  • [2025/07] Get faster LLM inference and cheaper responses with LMCache and Redis (Redis blog).
  • [2025/07] LMCache extends its turbo-boost to multimodal models in vLLM V1 (blog).
  • [2025/06] LLM Production Stack goes cross-hardware: AMD, Arm and Ascend (blog).

About

LMCache is a KV cache management layer for LLM inference. It turns KV cache from a temporary state into reusable AI-native knowledge that can be stored persistently, reused across multiple serving engines, monitored with an observability stack, and transformed for better generation quality. As a result, LMCache reduces TTFT (time-to-first-token) and improves throughput, especially for long-context agentic, multi-turn conversation, and knowledge-augmented workloads (e.g., RAG).

LMCache is vendor-neutral. It can be used as a KV cache layer for a range of mainstream open-source serving engines, inference frameworks, hardware vendors, storage systems, and infrastructure providers. The vendor neutrality allows users to freely switch between serving engines and storage vendors, while reusing the stored KV caches.

LMCache Deployment Modes

Key features

  • Engine-independent deployment: LMCache, as a standalone daemon process, manages KV cache independently from the inference engine process, so that KV cache will not be lost even if the inference engine crashes (i.e., no fate-sharing with engines).

  • Persistent, tiered KV cache offloading and reuse: Move KV caches out of GPU memory into a tiered storage hierarchy spanning CPU memory, local storage, and remote backends, enabling reuse across requests, sessions, and engine instances to reduce repeated prefill computation and improve TTFT.

  • Production-level KV cache observability: LMCache provides a rich set of KV cache observability metrics, including typical Kubernetes metrics (health monitoring, performance diagnostics), KV-cache-specific metrics (request-level and token-level prefix cache hits, lifecycle, request-level KV cache performance), management metrics (user-specific usage), and more.

  • Pluggable storage and transport backends: Easily integrate remote storage and KV transfer backends through a unified interface, enabling KV cache offloading and sharing across storage providers. Through this interface, LMCache supports storage backends including CPU RAM, local disk (SSD), Redis/Valkey, Mooncake, InfiniStore, S3-compatible object storage, NIXL, and GDS.

  • Non-prefix KV reuse: Extend KV reuse beyond prefix caching by reusing cached KV blocks at any position in the prompt. This leverages CacheBlend to selectively recompute tokens for quality recovery.

  • PD disaggregation and KV transfer: Support KV cache transfer from prefill workers to decode workers over NVLink, RDMA, or TCP through transport layers such as NIXL.

  • Pluggable KV transformation: A simple interface for researchers to write compression, token dropping, and custom serialization through a flexible SERDE interface.

LMCache is becoming an integral layer in the LLM inference ecosystem, with community-driven integration with serving engines, inference frameworks, hardware vendors, storage systems, and infrastructure providers:

LMCache ecosystem

Getting Started

To use LMCache, simply install lmcache from your package manager, e.g. pip:

pip install lmcache

For more setup options and examples, see:

Contributing

We welcome and value contributions and collaborations. Join us in improving LMCache. Check out the Contributing Guide or join our Slack community to get started.

Adoption and Partnerships

LMCache has a growing community of developers, researchers, industry adopters, and partners building the next generation of efficient LLM inference systems.

LMCache Adoption and Partnerships

As an independent open-source project, LMCache is becoming the de-facto standard for KV Cache management in LLM inference. Its continued development and community work are supported in part by Tensormesh.

Citation

LMCache builds on research in KV cache management, including cache reuse, offloading, compression, and serving optimization. If you use LMCache in your research, please cite the LMCache paper and related work.

@article{cheng2025lmcache,
  title={LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference},
  author={Cheng, Yihua and Liu, Yuhan and Yao, Jiayi and An, Yuwei and Chen, Xiaokun and Feng, Shaoting and Huang, Yuyang and Shen, Samuel and Du, Kuntai and Jiang, Junchen},
  journal={arXiv preprint arXiv:2510.09665},
  year={2025}
}
Related papers 
@inproceedings{liu2024cachegen,
  title={Cachegen: Kv cache compression and streaming for fast large language model serving},
  author={Liu, Yuhan and Li, Hanchen and Cheng, Yihua and Ray, Siddhant and Huang, Yuyang and Zhang, Qizheng and Du, Kuntai and Yao, Jiayi and Lu, Shan and Ananthanarayanan, Ganesh and others},
  booktitle={Proceedings of the ACM SIGCOMM 2024 Conference},
  pages={38--56},
  year={2024}
}

@inproceedings{yao2025cacheblend,
  title={Cacheblend: Fast large language model serving for rag with cached knowledge fusion},
  author={Yao, Jiayi and Li, Hanchen and Liu, Yuhan and Ray, Siddhant and Cheng, Yihua and Zhang, Qizheng and Du, Kuntai and Lu, Shan and Jiang, Junchen},
  booktitle={Proceedings of the twentieth European conference on computer systems},
  pages={94--109},
  year={2025}
}

License

The LMCache codebase is licensed under Apache License 2.0. See the LICENSE file for details.

Project links

Data verified by PyPI on Aug 15, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.5.4)

3 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for apostaB from gravatar.com apostaB Avatar for deng451e from gravatar.com deng451e Avatar for LMCache from gravatar.com LMCache

Credits

Author: LMCache Team

GitHub Statistics

Data verified by PyPI on Aug 15, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python <3.14, >=3.10

Provides Extra

nixl

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lmcache-0.5.4rc3.tar.gz (8.2 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

lmcache-0.5.4rc3-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.3 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc3-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (16.2 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

lmcache-0.5.4rc3-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.3 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc3-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (16.2 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

lmcache-0.5.4rc3-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.3 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc3-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (16.1 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

lmcache-0.5.4rc3-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.2 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc3-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (16.0 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

File details

Details for the file lmcache-0.5.4rc3.tar.gz.

File metadata

  • Download URL: lmcache-0.5.4rc3.tar.gz
  • Upload date:
  • Size: 8.2 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lmcache-0.5.4rc3.tar.gz
Algorithm Hash digest
SHA256 d77fa7008559aa9af11c571df1325c7cfb900ac19905e5fe7124f28854987bfc
MD5 62ea0e93dc6fbfd7368635ea6d15062e
BLAKE2b-256 15c1e8d3116edc70a264eace850056c081035f2b35e98b6d76913788889bddcf

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc3.tar.gz:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc3-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc3-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 8892d17c47ee1dc3ca0f4429438b79a5d588aa452ab44f8be5fb3567e68ca797
MD5 d9ccfddf6a130633bed43a4a118e1fca
BLAKE2b-256 8b4b7a9ab1705d49f973841741e34affde9e75e0035da4b298986028aa489285

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc3-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc3-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc3-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 3edaec81bd6d6ca0cb1616211c4bd1576f964663b5cd455ea0bd676e5b904af8
MD5 f1738a1e72b284498caf62f44228c30d
BLAKE2b-256 148260cdcdc62f7be6911f3b47f3410929df7c995182b0ab219a1173f98c6752

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc3-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc3-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc3-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 e91ab4c53d146794388c95591eb4eb9f780b698cfcbdc3391899b69c45f16667
MD5 747197a2519de11ff7e6779aa4aa0d70
BLAKE2b-256 7448129656a27fa308cacd516fcf351a42a101112b74bb9d339ffef2fcbaede6

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc3-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc3-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc3-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 e95ec2a041fc55edce14be57bb1a1aec517857471ab0e76de511848f167e3cd2
MD5 699b8f09e2ee800af2e45ecdc7a97899
BLAKE2b-256 6e007d92e0b168a9cd164f8bbe948ec28065d42ca94a64ef4f035b66fcffdc4b

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc3-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc3-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc3-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 334270821ef0c8fa74b57c25a2985e5b7804173050cfb53060608d7f6b316f01
MD5 cb18e4d9c8602d7ca56155cdd3233709
BLAKE2b-256 15a7fa8a84efa0668c2a6a2770b7eb47c35996786720aa66c3b46c1516c72b24

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc3-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc3-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc3-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 3993786eab6ef38440fe612efb7324eb8e4b60990f007cde5f4169f29fc92477
MD5 6e00e4c4572be063a72b35a9c6b1a32b
BLAKE2b-256 ff1ca75b464e7abbddb0d2dd12e866154f639ddc922bfdd11f6531af199b91ad

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc3-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc3-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc3-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 0f5db2c4763279b509faf5617eb45d97d94956aff16d8bbc80867f46b341c6ff
MD5 8ecfd58268c107cbbc578d4e265e8724
BLAKE2b-256 d7563cc2bfeecf83bf2082933b8cb10b7510a1fd7be2b572e82c27e592a26ea2

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc3-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc3-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc3-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 c2e3c8c18a691f57bd4bc1d7ec65f7aa6d03750ecb41352a83c3ba64b064b0f1
MD5 26d53699b32933ed369d693f55e75405
BLAKE2b-256 700c21bd8107bd3fb2396557b7566a2c2486b9f256243e5ba4ce3ad13838011d

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc3-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page