open-titans 0.0.5

Memory-Augmented Sequence Models in Pytorch

🌌 OpenTitans

The Open-Source Framework for Memory-Augmented Sequence Models

---

Democratizing Test-Time Memorization and Neural Memory Architectures.

Introduction • Documentation • Features • Quick Start • Citations

🌟 Introduction

OpenTitans is a modular, high-performance framework designed to implement and explore the next generation of sequence models. While Transformers revolutionized AI, their quadratic context limitations have met their match.

Inspired by groundbreaking research from Google and other top labs, OpenTitans focuses on Memory-Augmented Models that learn to memorize, optimize, and cache their internal states at test time. Our goal is to provide a "HuggingFace-like" experience for researchers and engineers building the future of infinite-context modeling.

---

📖 Documentation

For detailed information on how to use OpenTitans, please refer to our Documentation Index.

• Installation Guide: Setup and dependency management.
• Quickstart: Your first model in 5 minutes.
• Titans Variants: Understanding MAC, MAG, and MAL.
• ATLAS Framework: Learning to Optimally Memorize Context at Test Time.
• Neural Memory: Customizing memory models and fast-weight updates.

---

📦 Quick Start

Installation

[!TIP] We recommend using a virtual environment (venv or conda) for the best experience.

pip install open-titans

Development Installation

# Clone the repository
git clone https://github.com/Neeze/OpenTitans.git
cd OpenTitans

# Install in editable mode with dependencies
pip install -e .

---

🤝 Contributing

We are looking for "Titans" to help us build! 🚀

Whether you want to implement a new paper, optimize a CUDA kernel, or just fix a typo, your contributions are welcome. Check out our CONTRIBUTING.md to get started.

---

📚 Citations & Acknowledgements

OpenTitans stands on the shoulders of giants. We acknowledge the authors of the following papers for their foundational work:

@misc{behrouz2024titanslearningmemorizetest,
      title={Titans: Learning to Memorize at Test Time}, 
      author={Ali Behrouz and Peilin Zhong and Vahab Mirrokni},
      year={2024},
      url={https://arxiv.org/abs/2501.00663}
}

@misc{behrouz2025itsconnectedjourneytesttime,
      title={It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization}, 
      author={Ali Behrouz and Meisam Razaviyayn and Peilin Zhong and Vahab Mirrokni},
      year={2025},
      url={https://arxiv.org/abs/2504.13173}
}

@misc{behrouz2025atlaslearningoptimallymemorize,
      title={ATLAS: Learning to Optimally Memorize the Context at Test Time}, 
      author={Ali Behrouz and Zeman Li and Praneeth Kacham and Majid Daliri and Yuan Deng and Peilin Zhong and Meisam Razaviyayn and Vahab Mirrokni},
      year={2025},
      url={https://arxiv.org/abs/2505.23735}
}

@misc{behrouz2025nestedlearningillusiondeep,
      title={Nested Learning: The Illusion of Deep Learning Architectures}, 
      author={Ali Behrouz and Meisam Razaviyayn and Peilin Zhong and Vahab Mirrokni},
      year={2025},
      eprint={2512.24695},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2512.24695}, 
}

@misc{behrouz2026memorycachingrnnsgrowing,
      title={Memory Caching: RNNs with Growing Memory}, 
      author={Ali Behrouz and Zeman Li and Yuan Deng and Peilin Zhong and Meisam Razaviyayn and Vahab Mirrokni},
      year={2026},
      eprint={2602.24281},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2602.24281}, 
}

---

📄 License

OpenTitans is released under the MIT License. See LICENSE for more details.