transcoders-slim

Quickstart

Installation

pip install transcoders-slim

Usage

Load pre-trained transcoders from Jacob Dunefsky's repository: https://github.com/jacobdunefsky/transcoder_circuits

from transcoders.transcoder import Transcoder
from transcoders_slim.load_pretrained import load_pretrained
transcoders: dict[str, Transcoder] = load_pretrained()
for name, transcoder in transcoders.items():
    print(name, transcoder)

Run a transcoder:

d_in = transcoder.d_in
d_tr = transcoder.d_sae
seq_len = 32
tr_in = torch.zeros(1, seq_len, d_in).to(transcoder.device)
tr_out, tr_hid = transcoder(tr_in)[:2]

Development

Refer to Setup for how to set up development environment.