Pre-release

This release is a pre-release and may not be stable for production use.

lmcache logo

A KV Cache Management Layer for Scalable LLM Inference

Blog | Documentation | Join Slack | Community Meeting | Roadmap

GitHub Repo stars PyPI PyPI - Downloads GitHub commit activity GitHub contributors Good first issues Slack Ask DeepWiki

⭐ If LMCache helps you serve LLMs faster and cheaper, give us a star — it helps more teams discover the project.

Updates

  • [2026/05] 🔥 Agentic workload benchmark on AMD MI300X (blog).
  • [2026/04] 🔥 LMCache's new multiprocess (MP) architecture release (blog).
  • [2026/03] LMCache at GTC 2026 (post).
  • [2026/01] LMCache multi-node P2P CPU memory sharing, from experimental feature to production (blog).
More
  • [2025/11] LMCache x CoreWeave accelerate efficient LLM inference for Cohere (blog).
  • [2025/10] LMCache joins the PyTorch Foundation and Tensormesh unveiled (blog, PyTorch).
  • [2025/09] NVIDIA Dynamo integrates LMCache, accelerating LLM inference (blog).
  • [2025/08] 🎉 LMCache hits 5,000+ GitHub stars (blog).
  • [2025/08] LMCache supports gpt-oss (20B/120B) on day 1 (blog).
  • [2025/07] Get faster LLM inference and cheaper responses with LMCache and Redis (Redis blog).
  • [2025/07] LMCache extends its turbo-boost to multimodal models in vLLM V1 (blog).
  • [2025/06] LLM Production Stack goes cross-hardware: AMD, Arm and Ascend (blog).

About

LMCache is a KV cache management layer for LLM inference. It turns KV cache from a temporary state into reusable AI-native knowledge that can be stored persistently, reused across multiple serving engines, monitored with an observability stack, and transformed for better generation quality. As a result, LMCache reduces TTFT (time-to-first-token) and improves throughput, especially for long-context agentic, multi-turn conversation, and knowledge-augmented workloads (e.g., RAG).

LMCache is vendor-neutral. It can be used as a KV cache layer for a range of mainstream open-source serving engines, inference frameworks, hardware vendors, storage systems, and infrastructure providers. The vendor neutrality allows users to freely switch between serving engines and storage vendors, while reusing the stored KV caches.

LMCache Deployment Modes

Key features

  • Engine-independent deployment: LMCache, as a standalone daemon process, manages KV cache independently from the inference engine process, so that KV cache will not be lost even if the inference engine crashes (i.e., no fate-sharing with engines).

  • Persistent, tiered KV cache offloading and reuse: Move KV caches out of GPU memory into a tiered storage hierarchy spanning CPU memory, local storage, and remote backends, enabling reuse across requests, sessions, and engine instances to reduce repeated prefill computation and improve TTFT.

  • Production-level KV cache observability: LMCache provides a rich set of KV cache observability metrics, including typical Kubernetes metrics (health monitoring, performance diagnostics), KV-cache-specific metrics (request-level and token-level prefix cache hits, lifecycle, request-level KV cache performance), management metrics (user-specific usage), and more.

  • Pluggable storage and transport backends: Easily integrate remote storage and KV transfer backends through a unified interface, enabling KV cache offloading and sharing across storage providers. Through this interface, LMCache supports storage backends including CPU RAM, local disk (SSD), Redis/Valkey, Mooncake, InfiniStore, S3-compatible object storage, NIXL, and GDS.

  • Non-prefix KV reuse: Extend KV reuse beyond prefix caching by reusing cached KV blocks at any position in the prompt. This leverages CacheBlend to selectively recompute tokens for quality recovery.

  • PD disaggregation and KV transfer: Support KV cache transfer from prefill workers to decode workers over NVLink, RDMA, or TCP through transport layers such as NIXL.

  • Pluggable KV transformation: A simple interface for researchers to write compression, token dropping, and custom serialization through a flexible SERDE interface.

LMCache is becoming an integral layer in the LLM inference ecosystem, with community-driven integration with serving engines, inference frameworks, hardware vendors, storage systems, and infrastructure providers:

LMCache ecosystem

Getting Started

To use LMCache, simply install lmcache from your package manager, e.g. pip:

pip install lmcache

For more setup options and examples, see:

Contributing

We welcome and value contributions and collaborations. Join us in improving LMCache. Check out the Contributing Guide or join our Slack community to get started.

Adoption and Partnerships

LMCache has a growing community of developers, researchers, industry adopters, and partners building the next generation of efficient LLM inference systems.

LMCache Adoption and Partnerships

As an independent open-source project, LMCache is becoming the de-facto standard for KV Cache management in LLM inference. Its continued development and community work are supported in part by Tensormesh.

Citation

LMCache builds on research in KV cache management, including cache reuse, offloading, compression, and serving optimization. If you use LMCache in your research, please cite the LMCache paper and related work.

@article{cheng2025lmcache,
  title={LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference},
  author={Cheng, Yihua and Liu, Yuhan and Yao, Jiayi and An, Yuwei and Chen, Xiaokun and Feng, Shaoting and Huang, Yuyang and Shen, Samuel and Du, Kuntai and Jiang, Junchen},
  journal={arXiv preprint arXiv:2510.09665},
  year={2025}
}
Related papers 
@inproceedings{liu2024cachegen,
  title={Cachegen: Kv cache compression and streaming for fast large language model serving},
  author={Liu, Yuhan and Li, Hanchen and Cheng, Yihua and Ray, Siddhant and Huang, Yuyang and Zhang, Qizheng and Du, Kuntai and Yao, Jiayi and Lu, Shan and Ananthanarayanan, Ganesh and others},
  booktitle={Proceedings of the ACM SIGCOMM 2024 Conference},
  pages={38--56},
  year={2024}
}

@inproceedings{yao2025cacheblend,
  title={Cacheblend: Fast large language model serving for rag with cached knowledge fusion},
  author={Yao, Jiayi and Li, Hanchen and Liu, Yuhan and Ray, Siddhant and Cheng, Yihua and Zhang, Qizheng and Du, Kuntai and Lu, Shan and Jiang, Junchen},
  booktitle={Proceedings of the twentieth European conference on computer systems},
  pages={94--109},
  year={2025}
}

License

The LMCache codebase is licensed under Apache License 2.0. See the LICENSE file for details.

Project links

Data verified by PyPI on Aug 15, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.5.4)

3 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for apostaB from gravatar.com apostaB Avatar for deng451e from gravatar.com deng451e Avatar for LMCache from gravatar.com LMCache

Credits

Author: LMCache Team

GitHub Statistics

Data verified by PyPI on Aug 15, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python <3.14, >=3.10

Provides Extra

nixl

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lmcache-0.5.4rc4.tar.gz (8.2 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

lmcache-0.5.4rc4-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.3 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc4-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (16.2 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

lmcache-0.5.4rc4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.3 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc4-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (16.2 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

lmcache-0.5.4rc4-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.3 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc4-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (16.1 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

lmcache-0.5.4rc4-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.2 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

lmcache-0.5.4rc4-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (16.0 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

File details

Details for the file lmcache-0.5.4rc4.tar.gz.

File metadata

  • Download URL: lmcache-0.5.4rc4.tar.gz
  • Upload date:
  • Size: 8.2 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lmcache-0.5.4rc4.tar.gz
Algorithm Hash digest
SHA256 f84d4b373f51273cb2985809e7dca51a34f13b6490850b06a492b6533f6708dc
MD5 1e6db54cef4dc209b652cf8ea4c29731
BLAKE2b-256 a56dd435c64877f5948f65bb9a9fd22313edc4d245961df8527b511092d424a0

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc4.tar.gz:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc4-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc4-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 ffc7b9c4b2fabfa5be85834fcc70acdaa3aef5f49a5e52ef5f3c492ff1dc1bec
MD5 56702b69f4e7aa32b53750a014792a44
BLAKE2b-256 aed4bb3e77ac954dc944592722ff99c5ef6df7198b6fcab8bcce7298fe1ec132

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc4-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc4-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc4-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 f1f1dc3319c213e017e0f573a363f0618b8008aa5c688459361b4333902ba344
MD5 aaa09b59f4cbc8d1a8ea81f9e6e340c7
BLAKE2b-256 571914376e332e2beafb7c83cc92985496b9e37422937381bb111188dd083a58

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc4-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 f3dbea39266024f9db4298fc269f2d105dae4d89ba31bf9d0b2702b3dd12c0b6
MD5 b60b04a4dabc75a7a6e7c86bf49b02fd
BLAKE2b-256 ee08eff8a858eb7305b4974645af53d883c0e33340f4962b4efd383fce47ad3e

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc4-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc4-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 9a11c9aa0ecfa3b66db845c7864a22397f9b13f5e7b923bf22f494a64b7ebc19
MD5 0fc4796c0a5e39c2cfe195f05fd7372c
BLAKE2b-256 bc395c029ec6a28e0ae754b1feced4218ac61d2c8f4e24af934a1c27a9818e13

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc4-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc4-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc4-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 5ca490157fcf6d4339fc9410d2e9fbb1db2cd9161a28d2929a55bdb9888391ae
MD5 8cc2cf9e94cf992760665c625982daaf
BLAKE2b-256 ba9313a56a53e04f91fb78856240a92df166f61a9a97735085522383a137e67c

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc4-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc4-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc4-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 2d3cec8b87bd055c7349bee298b12251371e074397b927d30e4ee7f4d4e002db
MD5 4be6065be3efeecb6fc692013ce0c889
BLAKE2b-256 47c6e7136c14960d8b1466dc25adeb4c71e7ced6ed86efaf6982f62a9a50ac13

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc4-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc4-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc4-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 f112344b1c1c2b282ade6b8821d598be22e1aa0388c9800a39507f3c936ae435
MD5 91fd3ff996224f5a5be3fd5e69774e79
BLAKE2b-256 94d5d63e2c486eb757e0d93e5865c6b4ab6faadab49644614d7cbb5ad380d096

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc4-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lmcache-0.5.4rc4-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for lmcache-0.5.4rc4-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 f61285ea5ed30630e1f0e3df72624d13657c8ca5950c09a7d7aadd2355960dd7
MD5 ea37da834f5364491bfb53f28803cf4c
BLAKE2b-256 a6404ef7fe65710fbadd3d8fff82b9ba04839b6f97ffab1c0c3ad5ea9443d246

See more details on using hashes here.

Provenance

The following attestation bundles were made for lmcache-0.5.4rc4-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish.yml on LMCache/LMCache

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.5.4

9 files

Pre-release

0.5.4rc5

9 files

This release
Pre-release

0.5.4rc4 This release

9 files

Pre-release

0.5.4rc3

9 files

Pre-release

0.5.4rc2

9 files

Pre-release

0.5.4rc1

9 files

0.5.3

5 files

Pre-release

0.5.3rc4

5 files

Pre-release

0.5.3rc2

5 files

Pre-release

0.5.3rc1

5 files

0.5.2

5 files

Pre-release

0.5.2rc1

5 files

0.5.1

5 files

Pre-release

0.5.1rc2

5 files

Pre-release

0.5.1rc1

5 files

0.5.0

5 files

Pre-release

0.5.0rc1

5 files

Pre-release

0.4.8rc4

1 file

Pre-release

0.4.8rc3

1 file

Pre-release

0.4.8.dev42

4 files

Pre-release

0.4.8.dev38

4 files

0.4.7

5 files

0.4.6

5 files

0.4.5

5 files

0.4.4

5 files

0.4.3

5 files

0.4.2

5 files

0.4.1

5 files

0.3.15

5 files

0.3.14

5 files

0.3.13

5 files

0.3.12

5 files

0.3.11

5 files

0.3.10.post2

5 files

0.3.10.post1

5 files

0.3.10

5 files

0.3.9.post2

5 files

0.3.9.post1

5 files

0.3.9

5 files

0.3.7

5 files

0.3.6

6 files

0.3.5

6 files

0.3.4

4 files

0.3.3

4 files

0.3.2

4 files

0.3.1.post1

4 files

0.3.1

4 files

0.3.0

1 file

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page