High-performance distributed key-value cache

Docs PyPI PyPI - Python Version PyPI - Downloads GitCode commit activity license


🔄Latest News

  • [2026/06] MemCache使能推理PrefixCache加速案例实践，请关注wiki主页案例库获取最新案例

  • [2025/12] MemCache已作为vllm-ascend backend使能大模型推理加速，详情查看vllm-ascend开源社区，使用示例

  • [2025/11] MemCache项目于2025年11月开源，开源社区地址为：https://gitcode.com/Ascend/memcache

🔜 Roadmap&发布策略

MemCache roadmap详见： Roadmap MemCache 分支发布策略：分支发布策略

🎉概述

MemCache是针对LLM推理、GR推理场景设计的高性能分布式KVCache存储引擎，其主要特性包括：

  • 基于对象操作的API：支持批量和非批量的put/get/exist/remove操作，支持多层结构的KV Block读写接口。
  • 多层缓存池：支持HBM，DDR，SSD组成多级缓存池，多级之间通过淘汰和预取进行数据冷热交换。
  • 高带宽低时延：使用 MemFabric 作为多级内存和异构网络传输的底座，在Ascend硬件上，基于device_rdma(A2)、device_sdma(A3)、host_rdma(A2/A3)、device_urma(A5)、device_uboe(A5) 等路径提供OneCopy跨机跨介质数据直接访问能力，满足高带宽，低时延的读写性能述求。在鲲鹏硬件上，支持host_urma(K5)。支持host_shm实现同节点共享内存通信。
  • 支持扩缩容：支持LocalService动态加入和移除
  • HA能力：在K8S集群中，MetaService支持多活能力，支持元数据恢复，提供尽力而为的HA能力。
architecture

🧩核心组件

MemCache包含LocalService和MetaService两大核心组件：

  • MetaService：

    • 负责管理整个集群中内存池空间的分配和管理，处理LocalService的加入与退出。
    • MetaService作为独立进程运行，提供两种启动方式：python API启动；二进制启动，详见 whl安装使用 和 run安装使用
    • MetaService支持两种部署形态： 1、单点模式：MetaService由单个进程组成，部署方式简单，但存在单点故障的问题。如果MetaService进程崩溃或无法访问，系统将无法继续提供服务，直至重新恢复为止。 2、HA模式：该模式基于K8S的的ClusterIP Service和Lease资源构建，部署较为复杂，该模式会部署多个MetaService进程实例，实现多活高可用。部署详见怎么部署一个MemCache的HA集群

  • LocalService：负责承担如下功能：

    • 客户端：作为客户端，以whl/so形式作为共享库被应用进程加载调用API
    • 内存提供者：负责提供一段连续的内存区域作为内存池空间的一部分，其内存可以被其他LocalService实例基于地址直接访问。

🔥性能表现

MemCache核心能力是提供大容量内存池和高性能的H2D、D2H、D2RH、RH2D数据访问能力，由于MemCache以 MemFabric 作为池化底座，所以支持RH2D、D2RH等OneCopy跨机跨介质数据直接访问能力，下图为RH2D对比其他中转路径的对比示意图。

RH2D

基于OneCopy跨机跨介质数据直接访问的能力，MemCache在A2/A3做了相关性能测试如下： 模拟构造DeepSeek-R1模型KV大小的block，单个block size为：61x128K + 61x16K = 8784KB ≈ 8.57MB，共122个离散地址。

  • 使用2个昇腾A2节点(每节点8张卡)组成双机内存池进行读写测试性能如下：
A2 perf
  • 使用2个昇腾A3节点(每节点8张卡16Die)组成双机内存池进行读写测试性能如下：
A3 perf

🚀快速入门

请访问以下文档获取简易教程。

📑学习教程

  • C++接口：C++接口介绍以及C++接口对应的API列表
  • Python接口：Python接口介绍以及Python接口对应的API列表
  • RESTful接口：指标监控接口介绍以及API列表

📦软件硬件配套说明

  • MemCache软件依赖 MemFabric，相关配套与MemFabric相同

📌FAQ

常见问题请参考：FAQ

📝相关信息

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for monkeylove from gravatar.com monkeylove

License

Mulan PSL v2

Requires

Python >=3.8

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

memcache_hybrid-1.2.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (13.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

memcache_hybrid-1.2.0-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (12.3 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

memcache_hybrid-1.2.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (13.4 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

memcache_hybrid-1.2.0-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (12.3 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

memcache_hybrid-1.2.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (13.4 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

memcache_hybrid-1.2.0-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (12.3 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

memcache_hybrid-1.2.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (13.4 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

memcache_hybrid-1.2.0-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl (12.3 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.26+ ARM64manylinux: glibc 2.28+ ARM64

File details

Details for the file memcache_hybrid-1.2.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for memcache_hybrid-1.2.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 04f3e3913588004792445c8a9e58dae89155274c4606daa87afb000e5228bbdc
MD5 81283dc2d4bdb86b91dc6919707647fd
BLAKE2b-256 f78087ea91b2aba2a9f7c1fd755c1493615af5a4d71a76fea9bff128313a6fce

See more details on using hashes here.

File details

Details for the file memcache_hybrid-1.2.0-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for memcache_hybrid-1.2.0-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 6469161d01e77dcd1cef4c40a7707b34897d4a2f0283e206b47945d6d8aadd27
MD5 e14dbb091beb187df6c696ff9a0f05ba
BLAKE2b-256 5a4913262aa9b129a48b3628a60e3707a283d77273bb6a775c524a67aae15eac

See more details on using hashes here.

File details

Details for the file memcache_hybrid-1.2.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for memcache_hybrid-1.2.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 ada27d3d50ee6e788d2f642dc3dbe31e571e8346f7286cdcf5e05f522e6960a4
MD5 b59b345b305c9ee7b249bd45386929dd
BLAKE2b-256 de33fb5af44da501229ea4fc63626482ae27f1c746d64d902483f254dd1f4255

See more details on using hashes here.

File details

Details for the file memcache_hybrid-1.2.0-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for memcache_hybrid-1.2.0-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 b019a74c46c7cf3307276e50664725de8725daa0672bbb2715e6eaa76daacc39
MD5 234718a33ec13754f16f06ec4088c527
BLAKE2b-256 a33ff3276b57a648a4112f2e131816ded5a070ad563c5b10671877a158b76928

See more details on using hashes here.

File details

Details for the file memcache_hybrid-1.2.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for memcache_hybrid-1.2.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 c897aadd8f39fcbd79c66ad219ed51ed58787c52ae192edbd2e87c669afa3e01
MD5 8790731227f682ce284a184544571cc6
BLAKE2b-256 16f85a07a4a1824171abefc86c697d7b65b04771194d51b6cf19cf9bc9d10b73

See more details on using hashes here.

File details

Details for the file memcache_hybrid-1.2.0-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for memcache_hybrid-1.2.0-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 26fb1e0fe8998a402b1fccc114f174df56ae1bf2ee5860fb88e001546cb92bc9
MD5 c594752772fb966489d62884c32df6a8
BLAKE2b-256 dc91d4d9f2307999446120ce3fba3dd3ca266f461a96181a42352ecde38a316a

See more details on using hashes here.

File details

Details for the file memcache_hybrid-1.2.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for memcache_hybrid-1.2.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 606489559de754e156031edee1f3ee96d6320587ba490df17bf27473204bc4e8
MD5 87f2294b37874cade0509636c0c55b00
BLAKE2b-256 e690b60fd7262a95ad9a3581bf8e7f91c5992b46d835577c439d43b52610ed85

See more details on using hashes here.

File details

Details for the file memcache_hybrid-1.2.0-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for memcache_hybrid-1.2.0-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 dba9afd50acd7244414520eebd12b00ea705a9375af1bfcaf3fb3ed150f77741
MD5 3a4bddabedfbbd9d72ec053c01b441b5
BLAKE2b-256 fe0e88aecdee19d93cbf228df080ef105fd8929602410cb6453ea52bf430cbe2

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

1.2.0 This release

8 files

1.1.5

8 files

1.1.4

8 files

1.1.3

8 files

1.1.2

8 files

1.1.1

8 files

1.1.0

8 files

1.0.4

6 files

1.0.3

2 files

1.0.2

2 files

1.0.1

2 files

1.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page