MambaByte - Pytorch
Project description
MambaByte
Implementation of MambaByte in "MambaByte: Token-free Selective State Space Model" in Pytorch and Zeta. Note this will be a higher performance implementation of Mamba with parallel scan
Installation
pip install mambabyte
Usage
import torch
from mambabyte import MambaConfig, Mamba
x = torch.randn(2, 3, 4)
config = MambaConfig(
dim = 4,
depth = 3,
dt_rank = 2,
d_state = 2,
expand_factor = 2,
d_conv = 3,
dt_min = 0.001,
dt_max = 0.1,
dt_init = "random",
dt_scale = 1.0,
bias = False,
conv_bias = True,
pscan = True
)
model = Mamba(config)
out = model(x)
print(out)
License
MIT
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
mambabyte-0.0.2.tar.gz
(7.0 kB
view hashes)
Built Distribution
Close
Hashes for mambabyte-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1545a1276449a9d5cb3a0f3ee65779a6b6a48e701699e734a95822d22c9f59cb |
|
MD5 | c787007ccbb1294eee1f8e716dc24dd5 |
|
BLAKE2b-256 | 06a7a79de0f8b0e09fc430f168a222946d50f5d7842b07655d7865b67dd22272 |