mempalace-gpu

GPU-accelerated fork of milla-jovovich/mempalace

This fork adds GPU-accelerated embeddings and batch processing to MemPalace. Supports NVIDIA (CUDA), AMD (ROCm), and Apple Silicon (MPS). For documentation on MemPalace itself (palace structure, AAAK dialect, MCP tools, benchmarks), see the upstream README.

What this fork adds

GPU-accelerated embeddings

Embeddings are computed via sentence-transformers on GPU when available, falling back to ChromaDB's default CPU/ONNX model when not.

mempalace mine ~/myproject --device auto    # auto-detect best GPU
mempalace mine ~/myproject --device cuda    # NVIDIA
mempalace mine ~/myproject --device rocm    # AMD
mempalace mine ~/myproject --device mps     # Apple Silicon (M1-M5)
mempalace mine ~/myproject --device cpu     # force CPU

Also configurable via MEMPALACE_DEVICE env var or "device" in ~/.mempalace/config.json.

Batch processing

collection.add() calls are batched (100 documents per call instead of 1), reducing ChromaDB overhead regardless of CPU or GPU mode.

Self-update MCP tool

The MCP server includes a mempalace_self_update tool that pulls the latest version from PyPI, callable directly from your AI assistant.

Performance

Tested on two real-world codebases on NVIDIA CUDA. Same files, same drawers — only the device changes.

Speedup scales with drawer count. More chunks = more embedding work = bigger GPU advantage.

Installation

# Clone this fork
git clone https://github.com/phobicdotno/mempalace-gpu.git
cd mempalace-gpu

NVIDIA (CUDA)

pip install -e ".[gpu]"

AMD (ROCm)

# Install PyTorch with ROCm first
pip install torch --index-url https://download.pytorch.org/whl/rocm6.2
# Then install mempalace with GPU extras
pip install -e ".[gpu]"

Apple Silicon (MPS)

# PyTorch ships with MPS support on macOS by default
pip install -e ".[gpu]"

CPU only (still gets batch processing)

pip install -e .

Staying in sync with upstream

git remote add upstream https://github.com/milla-jovovich/mempalace.git
git fetch upstream
git merge upstream/main

Changes from upstream

All other files are unmodified from upstream. Existing palaces remain compatible.

License

MIT -- same as upstream.