TransformerLens
A Library for Mechanistic Interpretability of Generative Language Models. Maintained by Bryce Meyer and Jonah Larson; created by Neel Nanda
This is a library for doing mechanistic interpretability of GPT-2 Style language models. The goal of mechanistic interpretability is to take a trained model and reverse engineer the algorithms the model learned during training from its weights.
TransformerLens lets you load in 15,000+ open source language models across 140+ architecture families, and exposes the internal activations of the model to you. You can cache any internal activation in the model, and add in functions to edit, remove or replace these activations as the model runs.
Quick Start
Install
pip install transformer_lens
Python 3.8 or 3.9
pip install 'transformer_lens~=2.0'
Use
from transformer_lens.model_bridge import TransformerBridge
# Load a model (eg GPT-2 Small)
bridge = TransformerBridge.boot_transformers("gpt2", device="cpu")
# Run the model and get logits and activations
logits, activations = bridge.run_with_cache("Hello World")
Gated models (Llama, Mistral, Gemma, ...) require
HF_TOKENin your environment. See Environment Variables for the full list.
TransformerBridge is the recommended path and supports 15,000+ models across 140+ architecture families (see supported_models.json for the full inventory). By default it preserves raw HuggingFace weights – logits and activations match HF, not legacy HookedTransformer (which folds LayerNorm and centers weights by default). Call bridge.enable_compatibility_mode() after booting for HookedTransformer-equivalent numerics. The legacy HookedTransformer.from_pretrained API was removed in TransformerLens 4.0 — see the Migrating to TransformerLens 4.0 guide.
Key Tutorials
Gallery
Research done involving TransformerLens:
- Progress Measures for Grokking via Mechanistic Interpretability (ICLR Spotlight, 2023) by Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, Jacob Steinhardt
- Finding Neurons in a Haystack: Case Studies with Sparse Probing by Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, Dimitris Bertsimas
- Towards Automated Circuit Discovery for Mechanistic Interpretability by Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, Adrià Garriga-Alonso
- Actually, Othello-GPT Has A Linear Emergent World Representation by Neel Nanda
- A circuit for Python docstrings in a 4-layer attention-only transformer by Stefan Heimersheim and Jett Janiak
- A Toy Model of Universality (ICML, 2023) by Bilal Chughtai, Lawrence Chan, Neel Nanda
- N2G: A Scalable Approach for Quantifying Interpretable Neuron Representations in Large Language Models (2023, ICLR Workshop RTML) by Alex Foote, Neel Nanda, Esben Kran, Ioannis Konstas, Fazl Barez
- Eliciting Latent Predictions from Transformers with the Tuned Lens by Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, Jacob Steinhardt
User contributed examples of the library being used in action:
- Induction Heads Phase Change Replication: A partial replication of In-Context Learning and Induction Heads from Connor Kissane
- Decision Transformer Interpretability: A set of scripts for training decision transformers which uses transformer lens to view intermediate activations, perform attribution and ablations. A write up of the initial work can be found here.
Check out our demos folder for more examples of TransformerLens in practice
Getting Started in Mechanistic Interpretability
Mechanistic interpretability is a very young and small field, and there are a lot of open problems. This means there's both a lot of low-hanging fruit, and that the bar for entry is low - if you would like to help, please try working on one! The standard answer to "why has no one done this yet" is just that there aren't enough people! Key resources:
- A Guide to Getting Started in Mechanistic Interpretability
- ARENA Mechanistic Interpretability Tutorials from
Callum McDougall. A comprehensive practical introduction to mech interp, written in
TransformerLens - full of snippets to copy and they come with exercises and solutions! Notable
tutorials:
- Coding GPT-2 from scratch, with accompanying video tutorial from me (1 2) - a good introduction to transformers
- Introduction to Mech Interp and TransformerLens: An introduction to TransformerLens and mech interp via studying induction heads. Covers the foundational concepts of the library
- Indirect Object Identification: a replication of interpretability in the wild, that covers standard techniques in mech interp such as direct logit attribution, activation patching and path patching
- Mech Interp Paper Reading List
- 200 Concrete Open Problems in Mechanistic Interpretability
- A Comprehensive Mechanistic Interpretability Explainer: To look up all the jargon and unfamiliar terms you're going to come across!
- Neel Nanda's Youtube channel: A range of mech interp video content, including paper walkthroughs, and walkthroughs of doing research
Support & Community
If you have issues, questions, feature requests or bug reports, please search the issues to check if it's already been answered, and if not please raise an issue!
You're also welcome to join the open source mech interp community on Slack. Please use issues for concrete discussions about the package, and Slack for higher bandwidth discussions about eg supporting important new use cases, or if you want to make substantial contributions to the library and want a maintainer's opinion. We'd also love for you to come and share your projects on the Slack!
| ❗ HookedSAETransformer Removed |
|---|
Hooked SAE has been removed from TransformerLens in version 2.0. The functionality is being moved to SAELens. For more information on this release, please see the accompanying announcement for details on what's new, and the future of TransformerLens.
Mamba / SSM support (experimental)
TransformerLens includes bridge adapters for Mamba-1 (state-spaces/mamba-*-hf)
and Mamba-2 (AntonV/mamba2-130m-hf, state-spaces/mamba2-*, etc.). The adapters
cover:
- Forward pass (bit-for-bit HuggingFace equivalent)
- Hook-based introspection of projection activations (
in_proj,conv1d,x_proj,dt_proj,out_projfor Mamba-1;in_proj,conv1d,inner_norm,out_projfor Mamba-2) - Stateful generation with cache-aware decode steps
ActivationCache.compute_ssm_effective_attention(layer=...), which materializes the SSD-derived attention matrix of a Mamba-2 (or any other SSM / hybrid) layer for comparison with transformer attention patterns; the oldermamba2.compute_effective_attentionremains only as a deprecated alias
Verification lives in the integration tests at
tests/integration/model_bridge/test_mamba_adapter.py and
tests/integration/model_bridge/test_mamba2_adapter.py, and the
verify_models benchmark suite now covers the SSM and hybrid families. Mamba-1,
Mamba-2, gated-delta-net (Qwen3.5 / Qwen3-Next), NemotronH, and GraniteMoeHybrid
all declare applicable_phases = [1, 2, 3, 4], so their forward parity (P1, vs raw
HF), hook/cache self-checks and HF equivalence (P2/P3), and generation quality (P4)
are benchmarked like any transformer.
Credits
This library was created by Neel Nanda and is maintained by Bryce Meyer.
The core features of TransformerLens were heavily inspired by the interface to Anthropic's excellent Garcon tool. Credit to Nelson Elhage and Chris Olah for building Garcon and showing the value of good infrastructure for enabling exploratory research!
Creator's Note (Neel Nanda)
I (Neel Nanda) used to work for the Anthropic interpretability team, and I wrote this library because after I left and tried doing independent research, I got extremely frustrated by the state of open source tooling. There's a lot of excellent infrastructure like HuggingFace and DeepSpeed to use or train models, but very little to dig into their internals and reverse engineer how they work. This library tries to solve that, and to make it easy to get into the field even if you don't work at an industry org with real infrastructure! One of the great things about mechanistic interpretability is that you don't need large models or tons of compute. There are lots of important open problems that can be solved with a small model in a Colab notebook!
Citation
Please cite this library as:
@misc{nanda2022transformerlens,
title = {TransformerLens},
author = {Neel Nanda and Joseph Bloom},
year = {2022},
howpublished = {\url{https://github.com/TransformerLensOrg/TransformerLens}},
}
1 maintainer
Metadata
1 maintainer
Release files for transformer-lens 4.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| transformer_lens-4.2.0.tar.gz | 11.2 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| transformer_lens-4.2.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 12.8 MB
Release files / transformer_lens-4.2.0.tar.gz
| Download URL | transformer_lens-4.2.0.tar.gz |
|---|---|
| Size | 11.2 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
bde3753e0119ba11c90b21be46401d00927a1b3c35065bb5f6d7aec5eacfbebb
|
|
BLAKE2b-256 checksum How to use checksums |
b523bcbb5ccbd0b1e038681808a38a323eff1b37946bcf23cb44e7612796f623
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.0 {"installer":{"name":"uv","version":"0.13.0","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / transformer_lens-4.2.0-py3-none-any.whl
| Download URL | transformer_lens-4.2.0-py3-none-any.whl |
|---|---|
| Size | 1.6 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
202a54fcf8e12860805d064c60f59da093481eaa6435dc5775f97edff2607b21
|
|
BLAKE2b-256 checksum How to use checksums |
cca1a170e89d7f305276aebfab206cdd2534611e36164269632b06240457dd5c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.0 {"installer":{"name":"uv","version":"0.13.0","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
