Pre-release

This release is a pre-release and may not be stable for production use.

lmcache logo

A KV Cache Management Layer for Scalable LLM Inference

Blog | Documentation | Join Slack | Community Meeting | Roadmap

GitHub Repo stars PyPI PyPI - Downloads GitHub commit activity GitHub contributors Good first issues Slack Ask DeepWiki

⭐ If LMCache helps you serve LLMs faster and cheaper, give us a star — it helps more teams discover the project.

Updates

  • [2026/05] 🔥 Agentic workload benchmark on AMD MI300X (blog).
  • [2026/04] 🔥 LMCache's new multiprocess (MP) architecture release (blog).
  • [2026/03] LMCache at GTC 2026 (post).
  • [2026/01] LMCache multi-node P2P CPU memory sharing, from experimental feature to production (blog).
More
  • [2025/11] LMCache x CoreWeave accelerate efficient LLM inference for Cohere (blog).
  • [2025/10] LMCache joins the PyTorch Foundation and Tensormesh unveiled (blog, PyTorch).
  • [2025/09] NVIDIA Dynamo integrates LMCache, accelerating LLM inference (blog).
  • [2025/08] 🎉 LMCache hits 5,000+ GitHub stars (blog).
  • [2025/08] LMCache supports gpt-oss (20B/120B) on day 1 (blog).
  • [2025/07] Get faster LLM inference and cheaper responses with LMCache and Redis (Redis blog).
  • [2025/07] LMCache extends its turbo-boost to multimodal models in vLLM V1 (blog).
  • [2025/06] LLM Production Stack goes cross-hardware: AMD, Arm and Ascend (blog).

About

LMCache is a KV cache management layer for LLM inference. It turns KV cache from a temporary state into reusable AI-native knowledge that can be stored persistently, reused across multiple serving engines, monitored with an observability stack, and transformed for better generation quality. As a result, LMCache reduces TTFT (time-to-first-token) and improves throughput, especially for long-context agentic, multi-turn conversation, and knowledge-augmented workloads (e.g., RAG).

LMCache is vendor-neutral. It can be used as a KV cache layer for a range of mainstream open-source serving engines, inference frameworks, hardware vendors, storage systems, and infrastructure providers. The vendor neutrality allows users to freely switch between serving engines and storage vendors, while reusing the stored KV caches.

LMCache Deployment Modes

Key features

  • Engine-independent deployment: LMCache, as a standalone daemon process, manages KV cache independently from the inference engine process, so that KV cache will not be lost even if the inference engine crashes (i.e., no fate-sharing with engines).

  • Persistent, tiered KV cache offloading and reuse: Move KV caches out of GPU memory into a tiered storage hierarchy spanning CPU memory, local storage, and remote backends, enabling reuse across requests, sessions, and engine instances to reduce repeated prefill computation and improve TTFT.

  • Production-level KV cache observability: LMCache provides a rich set of KV cache observability metrics, including typical Kubernetes metrics (health monitoring, performance diagnostics), KV-cache-specific metrics (request-level and token-level prefix cache hits, lifecycle, request-level KV cache performance), management metrics (user-specific usage), and more.

  • Pluggable storage and transport backends: Easily integrate remote storage and KV transfer backends through a unified interface, enabling KV cache offloading and sharing across storage providers. Through this interface, LMCache supports storage backends including CPU RAM, local disk (SSD), Redis/Valkey, Mooncake, InfiniStore, S3-compatible object storage, NIXL, and GDS.

  • Non-prefix KV reuse: Extend KV reuse beyond prefix caching by reusing cached KV blocks at any position in the prompt. This leverages CacheBlend to selectively recompute tokens for quality recovery.

  • PD disaggregation and KV transfer: Support KV cache transfer from prefill workers to decode workers over NVLink, RDMA, or TCP through transport layers such as NIXL.

  • Pluggable KV transformation: A simple interface for researchers to write compression, token dropping, and custom serialization through a flexible SERDE interface.

LMCache is becoming an integral layer in the LLM inference ecosystem, with community-driven integration with serving engines, inference frameworks, hardware vendors, storage systems, and infrastructure providers:

LMCache ecosystem

Getting Started

To use LMCache, simply install lmcache from your package manager, e.g. pip:

pip install lmcache

For more setup options and examples, see:

Contributing

We welcome and value contributions and collaborations. Join us in improving LMCache. Check out the Contributing Guide or join our Slack community to get started.

Adoption and Partnerships

LMCache has a growing community of developers, researchers, industry adopters, and partners building the next generation of efficient LLM inference systems.

LMCache Adoption and Partnerships

As an independent open-source project, LMCache is becoming the de-facto standard for KV Cache management in LLM inference. Its continued development and community work are supported in part by Tensormesh.

Citation

LMCache builds on research in KV cache management, including cache reuse, offloading, compression, and serving optimization. If you use LMCache in your research, please cite the LMCache paper and related work.

@article{cheng2025lmcache,
  title={LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference},
  author={Cheng, Yihua and Liu, Yuhan and Yao, Jiayi and An, Yuwei and Chen, Xiaokun and Feng, Shaoting and Huang, Yuyang and Shen, Samuel and Du, Kuntai and Jiang, Junchen},
  journal={arXiv preprint arXiv:2510.09665},
  year={2025}
}
Related papers 
@inproceedings{liu2024cachegen,
  title={Cachegen: Kv cache compression and streaming for fast large language model serving},
  author={Liu, Yuhan and Li, Hanchen and Cheng, Yihua and Ray, Siddhant and Huang, Yuyang and Zhang, Qizheng and Du, Kuntai and Yao, Jiayi and Lu, Shan and Ananthanarayanan, Ganesh and others},
  booktitle={Proceedings of the ACM SIGCOMM 2024 Conference},
  pages={38--56},
  year={2024}
}

@inproceedings{yao2025cacheblend,
  title={Cacheblend: Fast large language model serving for rag with cached knowledge fusion},
  author={Yao, Jiayi and Li, Hanchen and Liu, Yuhan and Ray, Siddhant and Cheng, Yihua and Zhang, Qizheng and Du, Kuntai and Lu, Shan and Jiang, Junchen},
  booktitle={Proceedings of the twentieth European conference on computer systems},
  pages={94--109},
  year={2025}
}

License

The LMCache codebase is licensed under Apache License 2.0. See the LICENSE file for details.

Project links

Data verified by PyPI on Aug 20, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.5.4)

3 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for apostaB from gravatar.com apostaB Avatar for deng451e from gravatar.com deng451e Avatar for LMCache from gravatar.com LMCache

Credits

Author: LMCache Team

GitHub Statistics

Data verified by PyPI on Aug 20, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python <3.14, >=3.10

Provides Extra

nixl

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lmcache-0.5.4rc5.tar.gz (8.3 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

lmcache-0.5.4rc5-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.3 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc5-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (16.2 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

lmcache-0.5.4rc5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.3 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc5-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (16.2 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

lmcache-0.5.4rc5-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.3 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc5-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (16.1 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

lmcache-0.5.4rc5-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.2 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc5-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (16.1 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

File details

Details for the file lmcache-0.5.4rc5.tar.gz.

File metadata

  • Download URL: lmcache-0.5.4rc5.tar.gz
  • Upload date:
  • Size: 8.3 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lmcache-0.5.4rc5.tar.gz
Algorithm Hash digest
SHA256 f2ba5d68ddcd56810b610ae02dfd13fda3c3de4ad066e305cdd8cfac2b368065
MD5 978b02313245ca0622b91a0dcd887b35
BLAKE2b-256 5edc6d8da5505a8b97ba6691a10f39ef63ee50f6ac51a8fef095f69eb3dfbc74

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc5.tar.gz:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc5-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc5-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 77436677ef0b91728a7dc83632ea19d769fb1d50488ffd6fcf8d678721b1b860
MD5 1c7302eb35e9403b4f56bae4f9d94a54
BLAKE2b-256 c64f127a0b060b723e19bd327a78b1af703940972eb708d76ea72f3504745d7e

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc5-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc5-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc5-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 bae51726c6476bbc649aaffcc7ca1888c74a4d5372a5087298e372f8edbdc433
MD5 bdaa534c8078a1c4c24ebbecea25139e
BLAKE2b-256 6e2afd178d173c2fed312e04ab883020d680d5592adf5c987413076a74546080

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc5-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 1cf7c5c77162f2ed3732597dc99301d1bf9620d261b7f739b6257cb09345fd39
MD5 6a022adf8e58a52db85296179bf99d28
BLAKE2b-256 c8dcac7b45ee048d4c2638d1af1cec2258d78f727d32446062b755c5f3e8792b

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc5-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc5-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 09b1203828e51f17586a51d115c82e85282a4b0293ee38456afcd1afc2f55e31
MD5 2fe4f624abf9687fb6e22795f82ae776
BLAKE2b-256 b344318082827017cd6a0ed94eebeebd8e2758fb173892ec54b26bee3b484d50

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc5-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc5-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc5-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 d5825e9be8f2b49ecec96e920105a862d4d348a1edf7837f5ad458165a6a8c99
MD5 499872401473aac19ec2404edf54b675
BLAKE2b-256 06625a8dd28f67dd3d2d44614c8b02ace8e86aaabcfbd33076dc5a22df9cdd39

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc5-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc5-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc5-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 ecfd108bcc9b851ec7333697154cabae883e41e212f6946de3a223caf24203aa
MD5 97f7570ea40100f1f147c35d14c0eb49
BLAKE2b-256 f62c0357aaf0de57e1246379f3546a4c357a15c16fc1137e39ec8b412076324d

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc5-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc5-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc5-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 d0e155423dbef4428923b7f1240524d67cde5b942d6bc97b6566c9a6993932f7
MD5 0fc5590d30732d887f4f8ba85a62c6cb
BLAKE2b-256 2e5d43185f00d69e10775dbdd41d14bb0502f03a783100e2bfecc85c65ab69ee

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc5-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc5-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc5-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 30f41ba3bbe4771d9e207677df9e4882a57ecb2f871540ddbff7b3b17848dbcd
MD5 79e7043fabb549aaa6e421df2e7ca0d6
BLAKE2b-256 3e6aa0698ff5f3b6e9edcf37cc3d226a12771a3ea4ad983f5743589bfe276497

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc5-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.5.4

9 files

This release
Pre-release

0.5.4rc5 This release

9 files

Pre-release

0.5.4rc4

9 files

Pre-release

0.5.4rc3

9 files

Pre-release

0.5.4rc2

9 files

Pre-release

0.5.4rc1

9 files

0.5.3

5 files

Pre-release

0.5.3rc4

5 files

Pre-release

0.5.3rc2

5 files

Pre-release

0.5.3rc1

5 files

0.5.2

5 files

Pre-release

0.5.2rc1

5 files

0.5.1

5 files

Pre-release

0.5.1rc2

5 files

Pre-release

0.5.1rc1

5 files

0.5.0

5 files

Pre-release

0.5.0rc1

5 files

Pre-release

0.4.8rc4

1 file

Pre-release

0.4.8rc3

1 file

Pre-release

0.4.8.dev42

4 files

Pre-release

0.4.8.dev38

4 files

0.4.7

5 files

0.4.6

5 files

0.4.5

5 files

0.4.4

5 files

0.4.3

5 files

0.4.2

5 files

0.4.1

5 files

0.3.15

5 files

0.3.14

5 files

0.3.13

5 files

0.3.12

5 files

0.3.11

5 files

0.3.10.post2

5 files

0.3.10.post1

5 files

0.3.10

5 files

0.3.9.post2

5 files

0.3.9.post1

5 files

0.3.9

5 files

0.3.7

5 files

0.3.6

6 files

0.3.5

6 files

0.3.4

4 files

0.3.3

4 files

0.3.2

4 files

0.3.1.post1

4 files

0.3.1

4 files

0.3.0

1 file

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page