kachedb

High-Performance Python Client for KacheDB — The Zero-Copy Redis-Compatible & LLM KV-Cache Storage Engine

⚡ Installation

pip install kachedb

With PyTorch tensor zero-copy support:

pip install "kachedb[torch]"

With vLLM KV-cache acceleration plugin:

pip install "kachedb[vllm]"

Or install all extras:

pip install "kachedb[all]"

🚀 Quickstart

Synchronous Client

from kachedb import KacheClient

with KacheClient(host="127.0.0.1", port=6379) as client:
    # Standard Redis-compatible operations
    client.set("user:1", "alice", ex=3600)     # SET with 1-hour TTL
    print(client.get("user:1"))                 # b"alice"

    # Batch operations
    client.set("user:2", "bob")
    result = client.mget("user:1", "user:2")    # [b"alice", b"bob"]

    # Check existence
    print(client.exists("user:1"))              # 1

    # Delete
    client.delete("user:1", "user:2")

Async Client

import asyncio
from kachedb import AsyncKacheClient

async def main():
    async with AsyncKacheClient(host="127.0.0.1", port=6379) as client:
        await client.set("key", "value", ex=60)
        result = await client.get("key")
        print(result)  # b"value"

asyncio.run(main())

Pipeline Batching

Reduce network round-trips by batching multiple commands:

from kachedb import KacheClient

with KacheClient() as client:
    pipe = client.pipeline()
    pipe.set("a", "1")
    pipe.set("b", "2")
    pipe.set("c", "3")
    pipe.get("a")
    pipe.get("b")
    pipe.get("c")

    results = pipe.execute()
    # ["OK", "OK", "OK", b"1", b"2", b"3"]

Async Pipeline

from kachedb import AsyncKacheClient

async def main():
    async with AsyncKacheClient() as client:
        pipe = client.pipeline()
        pipe.set("x", "10")
        pipe.get("x")
        results = await pipe.execute()
        # ["OK", b"10"]

Zero-Copy Tensor Access (LLM KV-Cache)

Read KV-cache tensors directly from KacheDB's shared memory with zero data copying:

from kachedb import read_tensor, read_torch_tensor

# Read as numpy array (zero-copy via /dev/shm)
np_tensor = read_tensor(core_id=0, byte_offset=0)
print(np_tensor.shape, np_tensor.dtype)

# Read as PyTorch tensor (requires: pip install kachedb[torch])
torch_tensor = read_torch_tensor(core_id=0, byte_offset=0)
print(torch_tensor.shape, torch_tensor.dtype)

📋 Supported Commands

All commands follow the KacheDB RESP2/RESP3 wire protocol:

🏗️ Architecture

┌──────────────────────────────────────────────────────────┐
│                     Your Python App                      │
│             (vLLM / SGLang / FastAPI / etc.)             │
├──────────────────────────────────────────────────────────┤
│                   kachedb Python SDK                     │
│  ┌──────────────┐  ┌──────────────┐  ┌────────────────┐  │
│  │ KacheClient  │  │  AsyncKache  │  │    Pipeline    │  │
│  │  (sync TCP)  │  │    Client    │  │    Batching    │  │
│  └──────┬───────┘  └──────┬───────┘  └───────┬────────┘  │
│         │                 │                  │           │
│  ┌──────┴─────────────────┴──────────────────┴────────┐  │
│  │            RESP2/RESP3 Protocol Engine             │  │
│  │         (64KB buffered encoder + decoder)          │  │
│  └──────────────────────┬─────────────────────────────┘  │
│                         │                                │
│  ┌──────────────────────┴─────────────────────────────┐  │
│  │        ConnectionPool / AsyncConnectionPool        │  │
│  │    (Thread-safe / asyncio.Queue, health checks)    │  │
│  └──────────────────────┬─────────────────────────────┘  │
├─────────────────────────┼────────────────────────────────┤
│                   TCP + /dev/shm                         │
├──────────────────────────────────────────────────────────┤
│                  KacheDB Server (Rust)                   │
│         io_uring / kqueue │ POSIX SHM │ Megaslab         │
└──────────────────────────────────────────────────────────┘

🔧 Connection Pool

The client automatically manages a connection pool:

from kachedb import KacheClient

# Pool with up to 20 connections
client = KacheClient(
    host="127.0.0.1",
    port=6379,
    max_connections=20,
    socket_timeout=5.0,
)

🔌 vLLM KV-Cache Acceleration

KacheDB provides a plug-and-play connector for the vLLM distributed inference engine to accelerate prompt prefill and bypass redundant attention computation via zero-copy POSIX shared memory (/dev/shm):

1. Launch with vLLM CLI:

vllm serve meta-llama/Meta-Llama-3-8B-Instruct \
  --kv-transfer-config '{"kv_connector": "kachedb.vllm.KacheDBConnector", "kv_role": "kv_both"}'

2. Programmatic Usage in Custom Engines:

from kachedb.vllm import KacheDBConnector

# Initialize connector for worker rank
connector = KacheDBConnector(rank=0, local_rank=0, block_size=16)

# Restore cached prefix blocks directly into GPU PagedAttention buffers
matched_states, is_hit = connector.recv_kv_caches_and_hidden_states(
    model_executable=model,
    model_input=model_input,
    kv_caches=gpu_kv_caches,
)

🧪 Development

# Clone
git clone https://github.com/vubon/kachedb-py.git
cd kachedb-py

# Install in dev mode
pip install -e ".[dev]"

# Run unit tests
pytest tests/ -v --ignore=tests/test_integration.py

# Run integration tests (requires running KacheDB server)
pytest tests/test_integration.py -v

# Lint
ruff check src/ tests/
ruff format --check src/ tests/

# Type check
mypy src/kachedb/

🔗 Related Projects

• KacheDB Server — The Rust storage engine

📄 License

Dual-licensed under either of:

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT license (LICENSE-MIT)

at your option.