phantom-cache

Quantum-Unitary State Space Model (SSM) Prefix Caching, INT8 State Quantization, and Density Associative Memory.

⚡ Key Features

• Radix State Prefix Caching: 0ms prompt resumption for multi-turn chat & RAG (5.1x TTFT speedup on Mamba-130M).
• INT8 State Quantization: 4.0x memory compression (75% VRAM saved) with near-zero precision degradation (RMSE = 0.011).
• Quantum-Unitary Recurrence: Skew-Hermitian Cayley evolution operator (‖U‖₂ = 1.000) preventing exponential forgetting over 1,000,000+ tokens.
• Quantum Density Associative Memory: Outer-product density matrix storage with 100% exact Needle-in-a-Haystack recall across 1M words.
• PagedSSM & Multi-Tenant Router: Non-contiguous memory allocation with copy-on-write branching for enterprise inference serving.

📦 Installation

pip install phantom-cache

Or install from source:

git clone https://huggingface.co/Prannesshkva/Phantom-SSM-130M
cd Phantom-SSM-130M
pip install .

🚀 Quickstart

1. 0ms Prefix Caching with HuggingFace Mamba Models

from transformers import AutoModelForCausalLM, AutoTokenizer
from phantom_cache import SSMRadixStateCache, cache_prompt_prefix, ssm_cached_generate

model_id = "state-spaces/mamba-130m-hf"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# 1. Initialize INT8 Quantized Radix Cache
cache = SSMRadixStateCache(quantize_states=True, quant_mode="int8")

# 2. Pre-cache shared system prompt
sys_prompt = "You are a specialized AI assistant in physics.\n\n"
cache_prompt_prefix(model, tokenizer, sys_prompt, cache)

# 3. Instant 0ms prompt resumption
user_query = sys_prompt + "User: Explain the Hamiltonian operator."
response, stats = ssm_cached_generate(model, tokenizer, user_query, state_cache=cache)
print(f"Generated in {stats['total_s']:.3f}s (Reused {stats['matched_prefix']} prompt tokens!)")

2. Quantum Density Associative Memory

import torch
import torch.nn.functional as F
from phantom_cache import QuantumDensityAssociativeMemory

mem = QuantumDensityAssociativeMemory(key_dim=64, val_dim=64)
rho = mem.create_empty_state(batch_size=1)

# Write key-value pair
key = torch.randn(1, 64)
val = torch.randn(1, 64)
rho = mem.write(rho, key, val)

# Retrieve value using query key
retrieved = mem.read(rho, key)
similarity = F.cosine_similarity(retrieved, val, dim=-1)
print(f"Retrieval Cosine Similarity: {similarity.item():.5f}") # 1.00000

📜 License

Apache 2.0 License.