circuitkit 0.2.0

Discover, evaluate, and intervene on circuits in transformer models.
One call takes a model + task to a discovered circuit, a 6-pillar faithfulness score, and a pruned HuggingFace checkpoint.

CircuitKIT is a framework for mechanistic interpretability. Given a model and a task, it discovers the circuit driving that behaviour, evaluates how faithful it is, and lets you act on it (prune, quantize, edit, steer, or fine-tune), then export a reloadable HuggingFace checkpoint.

No GPU required for the quickstart — GPT-2 runs on CPU in a few minutes.

Quick start

# CPU-only, no GPU needed:
pip install -e .

# For benchmarking, add: pip install -e ".[benchmarks]"

from circuitkit import Pipeline

pipe = Pipeline("gpt2", task="ioi")
pipe.discover(algorithm="eap-ig", sparsity=0.3)
pipe.evaluate()
pipe.prune()
pipe.export("./checkpoint")

load_model, Pipeline and the api entry points also take a local checkpoint directory or any Hub repo id (e.g. a SafeTune or AlignTune output); the architecture is read from its config.json. Aya Vision and North Micro Vision are vision-language models and are rejected with an error: TransformerLens has no vision tower.

Single environment

CircuitKIT runs on transformer-lens 3.8.0 and transformers 5. Import circuitkit before transformer_lens, and a plain pip install circuitkit is enough.

import circuitkit as ck

model = ck.load_model("CohereLabs/aya-expanse-8b")  # or "CohereLabs/tiny-aya-global" (gated), or a local dir
circuit = ck.discover(model, "ioi", n_examples=32)
ck.export_checkpoint(model, circuit, "./aya-pruned")

ck.export_checkpoint writes the model's current weights, so ROME / MEMIT edits and weight steering are in the checkpoint, plus a lexsi_provenance.json. The *_scores.json files also carry safety_units / layer_suggestions, which SafeTune's CircuitKIT adapter reads.

Also works as a CLI and YAML config.

What is a circuit?

A circuit is the minimal set of attention heads and MLP layers in a transformer that drives a specific behaviour. Most interp tooling stops at "here is a subgraph with attribution scores." CircuitKIT goes further: it prunes (or quantizes) the model down to that subgraph, exports a reloadable HuggingFace checkpoint, and measures how faithful the pruned model stays. Because the circuit is task-specific, this produces a task-specialized checkpoint — not a general-purpose compressed model.

What you can do

Supported models

Production support covers Llama-3, Gemma/Gemma-3, Qwen, Mistral, Phi, Falcon, and GPT-2 (see Architecture Registry). CircuitKIT also supports five cohere-family and Llama-family models that TransformerLens doesn't natively support. Four of them (Tiny Aya, Command R7B, Aya Expanse 8B, SmolLM3-3B) are validated across all three surfaces — discovery, evaluation, and interventions. The fifth, Aya Expanse 32B, is registered, parity-validated (truncated depth), and checked for pruning and quantization module resolution (see below):

All of them except Aya Expanse 32B support discovery (5/6 stable algorithms plus parity), faithfulness evaluation, and interventions (pruning, quant resolution, steering), validated on real weights. Aya Expanse 32B is registered, with config conversion checked against the real checkpoint (truncated-depth parity) and module resolution for pruning scores and quantization targets checked on the full checkpoint. Full-depth parity, discovery, evaluation and weight steering do not run yet: the full 40-layer model needs about 64 GiB for weights alone, so it does not fit on most single GPUs, and transformer-lens 3.8.0's multi-GPU (n_devices) block placement is broken. See Experimental Models for what was and wasn't validated, and the details of the transformer-lens bug.

Two caveats on that support. acdc is excluded from the standard gate as impractically slow on 3–8B models — its test lives behind an opt-in flag — so the five validated on real weights are eap, eap-ig, eap-gp, ibcircuit, cdt. And "quant resolution" means target-module resolution confirmed on real weights; the actual optimum-quanto/llmcompressor compression call is not exercised in every environment, since those are optional dependencies.

Gemma-4 and Sarvam-MoE have separate experimental, discovery-only TransformerLens ports; this is not a claim of end-to-end CircuitKIT support. Their hardware and Sarvam loading requirements are documented in Experimental Models.

All five are experimental — see Tiny Aya and Experimental Models for architecture details, config mapping, and the full opt-in test matrix. Aya Expanse and Tiny Aya also load from local checkpoint directories and unlisted Hub repo ids.

Why CircuitKIT?

Next steps

Tests

pip install -e ".[dev]"
pytest tests/ -q

Citation

@software{circuitkit2026,
  title  = {CircuitKIT: Circuit Discovery, Evaluation, and Application Toolkit
            for Mechanistic Interpretability},
  author = {Seth, Pratinav and Gosalia, Hem and Kasliwal, Aditya
            and Sankarapu, Vinay Kumar},
  year   = {2026},
  url    = {https://github.com/Lexsi-Labs/circuitkit}
}

License

Lexsi Labs Source Available License (LSAL) v1.2: free for academic research and teaching on MIT-like terms; use by any organization requires written acknowledgement or permission (Section 1A); commercial use requires a separate license; responsible-use conditions apply. See LICENSE.md.