Attzoo 🔍

A modern, extensible PyTorch library for attention mechanisms in transformer models and deep learning architectures. Designed for educational and research purposes with clean, well-documented, and modular code.

📋 Attention Mechanisms

✅ Currently Implemented

🚀 Quick Start

Installation

# Install from source
git clone https://github.com/yourusername/attzoo.git
cd attzoo
uv sync

# For development
uv sync --all-extras --dev

Install from PyPI

Install the latest release from PyPI:

uv add attzoo

Notes:

• For a brand-new project on PyPI, use an account-scoped token for the first upload.
• Bump version in pyproject.toml before every release.

Basic Usage

import torch
from attzoo import MultiHeadSelfAttention

# Initialize model and input
d_model = 128
seq_len = 512
batch_size = 4

attn = MultiHeadSelfAttention(d_model=d_model, num_heads=8)
x = torch.randn(batch_size, seq_len, d_model)

# Forward pass
out, weights = attn(x)
print(out.shape)     # [4, 512, 128]
print(weights.shape) # [4, 8, 512, 512]

📝 Documentation

• Browse the generated docs locally:

  uv run -m mkdocs serve
  

  The site is served at http://127.0.0.1:8000/ with hot reload enabled.

• Produce a production build:

  uv run -m mkdocs build
  

  Static assets are written to site/ and published automatically by CI as part of the GitHub workflow.

📊 Performance Comparison

Where n=sequence length, w=window size, g=group ratio, d=dilation connections, b=number of blocks

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.