thelethe 0.0.1

AI models with adaptive memory management and strategic forgetting, inspired by Greek mythology and neuroscience.

theRiverLethe

AI models with adaptive memory management and strategic forgetting, inspired by Greek mythology and neuroscience.

Academic History of Titans Architecture

• Background: Prior to Titans Architecture, significant research was conducted related to test-time model adaptation and fast-weighted methods, primarily building on two key concepts:

  Self-supervised Learning and Meta Learning formed the foundation for these approaches.

• 2017-03-09 Model-Agnostic Meta-Learning (MAML): Pioneered fast adaptation to new tasks with minimal training examples, establishing core principles for adaptive models.
• 2019-09-29 Test-time Training: Advanced the field by introducing methods for model adaptation during inference time without requiring complete retraining.
• 2024-07-05 TTT-Linear/MLP: Introduced as a Self-Attention alternative for Transformers, enabling automatic retention of input sequences by using Self-supervised learning.
• 2024-12-31 Titans Architecture: Established a novel memory-based architecture combining Transformer's short-term memory capabilities (Self-Attention) with MLP-based long-term memory, significantly improving performance on extended sequential tasks.
• 2025-MM-DD Atlas Model

Overview

Titans Origin

• Titans introduced a new architecture family which consisted of:
  • Core Self-Attention (Short-term Memory, In-context learning)
  • Contextual Memory (Long-term Memory)
  • Persistent Memory (Fixed Memory)

Memory as a Context (MAC)

Memory as a Gate (MAG)

Memory as a Layer (MAL)

Atlas Model

• Atlas is a Titan in Greek mythology who is assigned the role of supporting the celestial sphere.

• Atlas Model proposes a combination of cognitive-scientific memory components.
  • Semantic Memory (Retrospective, Long-term Memory)
  • Episodic Memory (Retrospective, Long-term Memory)
  • Intentional Memory (Prospective, Long-term Memory)
  • Active Cognitive State (Working/Operational, Short-term Memory)