attnview

A notebook-first, dark HTML atlas for Transformer attention patterns.

attnview turns attention matrices into a focused research view: choose a layer, head, and query position; inspect the strongest destinations; then open the exact row or full matrix when you need to verify a detail. It is designed for mechanistic-interpretability work in Jupyter.

Install

pip install -U attnview

For notebooks, install the optional IPython extra when IPython is not already available:

pip install -U "attnview[notebook]"

Quick start

attention has shape [n_heads, query_length, key_length]. The first sequence axis is the query (destination) position; the second is the key (source) position.

import attnview

view = attnview.display(
    attention,
    tokens=tokens,
    title="My attention study",
)
view

The result is a self-contained HTML object rendered inline by Jupyter. Long prompts stay readable through a local token window and search, while the complete values remain available in the exact-matrix panel.

TransformerLens integration

Pass the ActivationCache returned by run_with_cache. attnview finds canonical blocks.{layer}.attn.hook_pattern entries, ignores unrelated activations, and combines all discovered layers into one atlas.

import attnview

tokens = model.to_tokens(prompt)
logits, cache = model.run_with_cache(tokens)

view = attnview.display_cache(
    cache,
    tokens=model.to_str_tokens(tokens),
)
view

Export the same report as a portable artifact:

attnview.export_cache(
    cache,
    "attention-atlas.html",
    tokens=model.to_str_tokens(tokens),
)

TransformerLens is not a runtime dependency. The adapter accepts any mapping-like cache exposing the same pattern keys, which makes saved fixtures and small experiments easy to inspect.

Cross-attention and explicit labels

For encoder-decoder or other cross-attention experiments, pass query and key labels separately:

attnview.display(
    cross_attention,
    query_tokens=decoder_tokens,
    key_tokens=encoder_tokens,
)

A reproducible research workflow

See the TransformerLens research guide for a complete notebook recipe, a small attention-concentration study, interpretation cautions, and primary-literature references.

API at a glance

Development

python -m pip install -e ".[dev,notebook]"
python -m pytest
python -m build
python -m twine check dist/*

Contributions and bug reports are welcome. Please read CONTRIBUTING.md and the Code of Conduct first.

License

MIT. See LICENSE.