Pre-release

This release is a pre-release and may not be stable for production use.

lmcache logo

A KV Cache Management Layer for Scalable LLM Inference

Blog | Documentation | Join Slack | Community Meeting | Roadmap

GitHub Repo stars PyPI PyPI - Downloads GitHub commit activity GitHub contributors Good first issues Slack Ask DeepWiki

⭐ If LMCache helps you serve LLMs faster and cheaper, give us a star — it helps more teams discover the project.

Updates

  • [2026/05] 🔥 Agentic workload benchmark on AMD MI300X (blog).
  • [2026/04] 🔥 LMCache's new multiprocess (MP) architecture release (blog).
  • [2026/03] LMCache at GTC 2026 (post).
  • [2026/01] LMCache multi-node P2P CPU memory sharing, from experimental feature to production (blog).
More
  • [2025/11] LMCache x CoreWeave accelerate efficient LLM inference for Cohere (blog).
  • [2025/10] LMCache joins the PyTorch Foundation and Tensormesh unveiled (blog, PyTorch).
  • [2025/09] NVIDIA Dynamo integrates LMCache, accelerating LLM inference (blog).
  • [2025/08] 🎉 LMCache hits 5,000+ GitHub stars (blog).
  • [2025/08] LMCache supports gpt-oss (20B/120B) on day 1 (blog).
  • [2025/07] Get faster LLM inference and cheaper responses with LMCache and Redis (Redis blog).
  • [2025/07] LMCache extends its turbo-boost to multimodal models in vLLM V1 (blog).
  • [2025/06] LLM Production Stack goes cross-hardware: AMD, Arm and Ascend (blog).

About

LMCache is a KV cache management layer for LLM inference. It turns KV cache from a temporary state into reusable AI-native knowledge that can be stored persistently, reused across multiple serving engines, monitored with an observability stack, and transformed for better generation quality. As a result, LMCache reduces TTFT (time-to-first-token) and improves throughput, especially for long-context agentic, multi-turn conversation, and knowledge-augmented workloads (e.g., RAG).

LMCache is vendor-neutral. It can be used as a KV cache layer for a range of mainstream open-source serving engines, inference frameworks, hardware vendors, storage systems, and infrastructure providers. The vendor neutrality allows users to freely switch between serving engines and storage vendors, while reusing the stored KV caches.

LMCache Deployment Modes

Key features

  • Engine-independent deployment: LMCache, as a standalone daemon process, manages KV cache independently from the inference engine process, so that KV cache will not be lost even if the inference engine crashes (i.e., no fate-sharing with engines).

  • Persistent, tiered KV cache offloading and reuse: Move KV caches out of GPU memory into a tiered storage hierarchy spanning CPU memory, local storage, and remote backends, enabling reuse across requests, sessions, and engine instances to reduce repeated prefill computation and improve TTFT.

  • Production-level KV cache observability: LMCache provides a rich set of KV cache observability metrics, including typical Kubernetes metrics (health monitoring, performance diagnostics), KV-cache-specific metrics (request-level and token-level prefix cache hits, lifecycle, request-level KV cache performance), management metrics (user-specific usage), and more.

  • Pluggable storage and transport backends: Easily integrate remote storage and KV transfer backends through a unified interface, enabling KV cache offloading and sharing across storage providers. Through this interface, LMCache supports storage backends including CPU RAM, local disk (SSD), Redis/Valkey, Mooncake, InfiniStore, S3-compatible object storage, NIXL, and GDS.

  • Non-prefix KV reuse: Extend KV reuse beyond prefix caching by reusing cached KV blocks at any position in the prompt. This leverages CacheBlend to selectively recompute tokens for quality recovery.

  • PD disaggregation and KV transfer: Support KV cache transfer from prefill workers to decode workers over NVLink, RDMA, or TCP through transport layers such as NIXL.

  • Pluggable KV transformation: A simple interface for researchers to write compression, token dropping, and custom serialization through a flexible SERDE interface.

LMCache is becoming an integral layer in the LLM inference ecosystem, with community-driven integration with serving engines, inference frameworks, hardware vendors, storage systems, and infrastructure providers:

LMCache ecosystem

Getting Started

To use LMCache, simply install lmcache from your package manager, e.g. pip:

pip install lmcache

For more setup options and examples, see:

Contributing

We welcome and value contributions and collaborations. Join us in improving LMCache. Check out the Contributing Guide or join our Slack community to get started.

Adoption and Partnerships

LMCache has a growing community of developers, researchers, industry adopters, and partners building the next generation of efficient LLM inference systems.

LMCache Adoption and Partnerships

As an independent open-source project, LMCache is becoming the de-facto standard for KV Cache management in LLM inference. Its continued development and community work are supported in part by Tensormesh.

Citation

LMCache builds on research in KV cache management, including cache reuse, offloading, compression, and serving optimization. If you use LMCache in your research, please cite the LMCache paper and related work.

@article{cheng2025lmcache,
  title={LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference},
  author={Cheng, Yihua and Liu, Yuhan and Yao, Jiayi and An, Yuwei and Chen, Xiaokun and Feng, Shaoting and Huang, Yuyang and Shen, Samuel and Du, Kuntai and Jiang, Junchen},
  journal={arXiv preprint arXiv:2510.09665},
  year={2025}
}
Related papers 
@inproceedings{liu2024cachegen,
  title={Cachegen: Kv cache compression and streaming for fast large language model serving},
  author={Liu, Yuhan and Li, Hanchen and Cheng, Yihua and Ray, Siddhant and Huang, Yuyang and Zhang, Qizheng and Du, Kuntai and Yao, Jiayi and Lu, Shan and Ananthanarayanan, Ganesh and others},
  booktitle={Proceedings of the ACM SIGCOMM 2024 Conference},
  pages={38--56},
  year={2024}
}

@inproceedings{yao2025cacheblend,
  title={Cacheblend: Fast large language model serving for rag with cached knowledge fusion},
  author={Yao, Jiayi and Li, Hanchen and Liu, Yuhan and Ray, Siddhant and Cheng, Yihua and Zhang, Qizheng and Du, Kuntai and Lu, Shan and Jiang, Junchen},
  booktitle={Proceedings of the twentieth European conference on computer systems},
  pages={94--109},
  year={2025}
}

License

The LMCache codebase is licensed under Apache License 2.0. See the LICENSE file for details.

Project links

Data verified by PyPI on Aug 12, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Stable release available (0.5.3)

3 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for apostaB from gravatar.com apostaB Avatar for deng451e from gravatar.com deng451e Avatar for LMCache from gravatar.com LMCache

Credits

Author: LMCache Team

GitHub Statistics

Data verified by PyPI on Aug 12, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python <3.14, >=3.10

Provides Extra

nixl

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lmcache-0.5.4rc1.tar.gz (8.1 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

lmcache-0.5.4rc1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc1-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (15.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

lmcache-0.5.4rc1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc1-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (15.4 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

lmcache-0.5.4rc1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (15.5 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc1-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (15.4 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

lmcache-0.5.4rc1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (15.5 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc1-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (15.3 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

File details

Details for the file lmcache-0.5.4rc1.tar.gz.

File metadata

  • Download URL: lmcache-0.5.4rc1.tar.gz
  • Upload date:
  • Size: 8.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lmcache-0.5.4rc1.tar.gz
Algorithm Hash digest
SHA256 0acf0833d4b1e7c11f607ff9cb216dcaf1dab0fc18fb9307c0b5ed6a4a5ded87
MD5 9569081acf83da3566f4b85a2ffe48ac
BLAKE2b-256 941f615cfdee631a6bf22fccf4764c30f3a10e07c0e31e35b1b9f183a81f3b25

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc1.tar.gz:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 7b4332ec9f05a07fa9f9f2170466fd58525f3777ea2763bba6be5beb9cdc6bd9
MD5 9ce35e44814881f09b891fbfa6d7c90a
BLAKE2b-256 574d5c7220ab2754068dea9ae2911e11b2e892236cf5aa5750fe399a2575e8ff

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc1-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc1-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 e6b48d4265e10f0160844296ba2d39132f9755e5369819f21196f933388e5fc4
MD5 b76f677daf33d4de3e23a0fe664be0b9
BLAKE2b-256 8819f2928dc3dc73ede4357447d0dac0a9f8777f74b2106a9c99825029ddc735

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc1-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 cfa331a7980ab32e3166bd828cbc9fe71f4af4f6b38a38880fa6ee850bbe2bc8
MD5 a31204f335ffe1dc9c9e81efdc407454
BLAKE2b-256 f9bbde0bff3180856bbc97024b76d9f5183d44a1c007cab2916382a65ece7f32

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc1-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc1-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 347d8df0560d8a975307f2da910e264f37d999334618804803f0d5d81b0f78df
MD5 537309397293627549ea82e50e73ad3f
BLAKE2b-256 a8dddac64a479b5005962f0eac5cf60239477666f6918cb25ba83112cd86a2f5

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc1-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 d4eba351c3f5c6aa0acdebab05d32cb1f7a96ddfd59673afe5e1c5cfd9f342ca
MD5 1a1050ad1bedbb63c3755b4a4e65c321
BLAKE2b-256 08a6782b754364c14b8cc3330ac2ec6f3b96b44f0cb1f01a404eaeaeec078cca

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc1-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc1-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 8937f93b4d78aa27d26b5a051b00725d727298e20d3e05ba64a93b66557da34e
MD5 6e166bb6caac173cb5d98f66555069bb
BLAKE2b-256 2ed07f39c3708d3d1c2b3b343d190876060de99a65baf298564385f162314615

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc1-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 412c7e71f7d190124f30745e2dd78b9d6a3cc85f30eedf3b558af48de42324be
MD5 1a634a03b29ff1b0c63a3d34a5a653cb
BLAKE2b-256 1d343b66ea5585b11f9d945b53d1d944fb89c922603dd0cdbcbce884026c6c8c

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc1-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc1-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 fc9419c0a8c6253293c3fdb423207b84c5460b5f58e83e24b858838cfefb9d4b
MD5 4fc4879abc10ed79bb4c3a1c5509b309
BLAKE2b-256 03bc0067df41e7fdcf7a1efa62a6d15d285061e981827373ff5c64a25e03dc1f

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc1-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page