belief-state-superposition 0.1.0

Investigating belief state representations of transformers trained on Hidden Markov Model emissions

belief-state-superposition

[]

Quickstart

Install

pip install belief-state-superposition

Usage

Generate and inspect data from a Hidden Markov Model

from belief_state_superposition.hmm import sample_sequence

data = sample_sequence(16)
beliefs, states, emissions, next_beliefs, next_states = zip(*data)
print(beliefs)
print(states)
print(emissions)

Train a model on belief states

import torch 
from torch.utils.data import DataLoader
from belief_state_superposition.model import init_model
from belief_state_superposition.data import get_dataset
from belief_state_superposition.train import train_model

device = "cuda" if torch.cuda.is_available() else "cpu"
train_dataset = get_dataset(1000)
train_data_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)
model = init_model().to(device)
train_model(model, train_data_loader, n_epochs=10, show_progress_bar=True, device = device)

Development

Refer to Setup for how to set up development environment.