LexFrame: Lexical Reference Frames

A toolkit for interpreting LM residual streams in their native semantics.

LexFrame reads a hidden state the way the model reads it, in the model's own vocabulary. It is model-native: the state is transported layer by layer through the model's own remaining layers, with nothing fitted to the output. It is context-aware: along the way the state attends to its prefix, for a chosen number of rounds of attention.

Quickstart

pip install lexframe

import lexframe as lf

frame = lf.LexFrame(model, tokenizer).init(texts)   # a loaded Hugging Face model and any unlabelled texts
c = frame.view("The Eiffel Tower is located in the city of", k=5)
c.tokens                                            # the top 5 tokens of every layer at every position

docs/USAGE.md lists every command, and demo.py walks through them: where SmolLM3 decides that the Eiffel Tower is in Paris, who wrote that decision, and how to change it.

Mechanism

A frame carries a hidden state through the model's own remaining layers, with the context the state attends to, and reads the result with the model's own unembedding. docs/MECHANISM.md explains the details.

Compatibility

LexFrame works well with any causal language model in 🤗 Transformers, including popular families such as Llama, Qwen, Gemma, Mistral and DeepSeek.

Citation

@article{gu2026lexframe,
  title   = {Lexical Reference Frames: Interpreting LM Residual Streams in Native Semantics},
  author  = {Gu, Jian and Zhang, Hongyu and Chen, Chunyang and Aleti, Aldeida},
  journal = {arXiv preprint arXiv:2610.22169},
  year    = {2026}
}

Licensed under the Apache License 2.0.