idempotent-hopfield: Zero-Copy Modern Hopfield Associative Memory & CAM Engine

Eliminate 100% of auxiliary VRAM allocations during Modern Hopfield Associative Memory pattern retrieval and Content-Addressable Memory (CAM) consolidation.

🚀 The Bottleneck: Associative Memory Reallocation

Modern Hopfield Networks (Ramsauer et al., 2020) and high-dimensional Content-Addressable Memory (CAM) architectures store massive banks of continuous prototype patterns to enable associative recall and energy-based pattern completion.

When retrieving and consolidating candidate memory states into stable attractor basins, conventional systems execute pattern compaction via out-of-place memory gathering (torch.gather / dynamic cudaMalloc):

1. Auxiliary Memory Bloat: Allocating secondary destination buffers of size $O(B \cdot K \cdot D)$ consumes hundreds of megabytes of transient VRAM across memory banks.
2. Memory Bus Saturation: Double-buffering redundant read/write cycles congests High-Bandwidth Memory (HBM).
3. Allocation Jitter: Dynamic memory management introduces latency spikes in high-throughput retrieval pipelines and vector search systems.

⚡ The Solution: In-Situ Idempotent Attractor Basin Locking

idempotent-hopfield rearranges continuous memory patterns directly within existing memory allocations using $O(1)$ scalar hardware registers:

• Idempotent Attractor Invariant: Enforces the algebraic property $f(f(x)) = f(x)$, locking retrieved patterns into stabilized contiguous attractor basins $[0, K-1]$.
• Bitmask-Free Cycle Follower: Resolves permutation orbits on GPU streaming multiprocessors without auxiliary bitmasks or auxiliary global memory.
• In-Register 2-Cycle Fast-Path: Mutually transposed memory patterns are swapped directly across thread registers with zero memory overhead.
• 100% Zero Auxiliary VRAM: Exactly 0.00 MB auxiliary secondary memory allocated.
• Bit-Exact Numerical Parity: Zero approximation error ($\Delta = 0.000000$, 0 NaN).
• Blazing Throughput: Exceeds 74+ Million patterns/second on modern NVIDIA GPUs.

📊 Benchmark: NVIDIA RTX PRO 500 Blackwell (sm_120)

Workload: 16 Memory Banks, 4,096 Patterns/Bank (65,536 total patterns), FeatureDim=512, Capacity=2,048 (50% Active, float16)

📦 Installation

git clone https://github.com/aemre-cetin/idempotent-hopfield.git
cd idempotent-hopfield
pip install -e .

Requirements: torch >= 2.0.0, triton >= 2.1.0.

🛠️ Quickstart

import torch
from idempotent_hopfield import InplaceHopfieldCAM

# Initialize CAM engine
cam = InplaceHopfieldCAM(hidden_dim=512, block_d=128)

# Memory pattern bank on GPU [Banks, Patterns, FeatureDim]
patterns = torch.randn((16, 4096, 512), dtype=torch.float16, device="cuda")

# Energy / associative recall affinity scores [Banks, Patterns]
energy_scores = torch.rand((16, 4096), dtype=torch.float32, device="cuda")

# In-place consolidation: locks top-2048 patterns into attractor basins with 0 bytes aux VRAM
consolidated_patterns = cam(patterns, energy_scores, capacity=2048)

# Output shape: [16, 2048, 512] directly physically contiguous
print("Consolidated patterns shape:", consolidated_patterns.shape)

PyTorch Hopfield Layer Hook

from idempotent_hopfield.integrations import HopfieldAssociativeLayerHook

# Drop-in hook for Modern Hopfield networks
hook = HopfieldAssociativeLayerHook(hidden_dim=512, capacity=2048)

# Execute in-place attractor consolidation
consolidated_memory = hook(memory_patterns, queries=query_embeddings)

🛡️ Patent & Intellectual Property Notice

The mathematical formulations, state-transition architectures, and in-situ hardware compaction kernels implemented in this library are protected under pending patent application with the United States Patent and Trademark Office:

• U.S. Patent Application Number: 64/148,668
• Confirmation Number: 5890
• Status: PATENT PENDING
• First Named Inventor: Dr. Ahmet Emre ÇETİN

Academic evaluation, non-commercial research, and open-source collaboration are permitted under the terms of the Apache 2.0 License. Commercial deployment in proprietary hardware or commercial cloud runtimes is subject to bilateral licensing agreements with the author.

📜 Academic Citation

@article{cetin2026idempotenthopfield,
  title={Zero-Copy Modern Hopfield Associative Memory and In-Situ Attractor Consolidation in Deep Learning Accelerators},
  author={Cetin, A. Emre},
  journal={arXiv preprint},
  year={2026},
  note={U.S. Patent Application No. 64/148,668}
}

@article{cetin2013idempotent,
  title={Idempotent Permutations},
  author={Cetin, A. E.},
  journal={arXiv:1307.3877 [cs.DS]},
  year={2013}
}

📄 License

Licensed under the Apache License, Version 2.0. Copyright © 2026 Dr. A. Emre ÇETİN. All Rights Reserved.