medtokenizers
medtokenizers compresses 2D and 3D medical images into latents or discrete
token grids. It provides one encoder-decoder backbone with interchangeable
quantization heads (VQ, LFQ, FSQ, Residual FSQ, VAE, AE), so the quantizer can
be treated as a controlled variable rather than a fixed preprocessing choice.
It also provides shared reconstruction metrics and dataset tokenization
utilities.
This library accompanies the NeurIPS 2026 paper Tokenizer-Generator Coupling in Medical Image Generation, which uses it for the tokenizer half of a factorial study. The paper's experiments live in tokenizer-generator-coupling. See Citation.
Install
pip install medtokenizers
Requires Python 3.10+ and PyTorch 2.0+. Tested on Linux and macOS. A CUDA GPU is needed for 3D training, but everything below runs on CPU.
For a CPU-only install, take PyTorch from its CPU index first so torch and
torchvision come from the same build:
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
pip install medtokenizers
To work on the library itself, install from a checkout instead:
git clone https://github.com/liamchalcroft/medtokenizers.git
cd medtokenizers
pip install -e ".[test]"
Run it
Swap the quantizer, keep everything else fixed:
import torch
from medtokenizers import DiscreteTokenizer
x = torch.randn(1, 1, 64, 64)
for quantizer in ["VQ", "FSQ", "RESFSQ"]:
model = DiscreteTokenizer(
dim=2, quantizer=quantizer, resolution=64, spatial_compression=8
).eval()
with torch.no_grad():
tokens = model.tokenize(x) # (1, 8, 8) integer codes
recon = model.detokenize(tokens) # (1, 1, 64, 64)
LFQ is the one head that needs two further arguments, because its codebook is defined by its binary dimension rather than inferred:
model = DiscreteTokenizer(
dim=2,
quantizer="LFQ",
resolution=64,
spatial_compression=8,
codebook_size=1024,
codebook_dim=10,
)
The continuous heads live on a separate class, selected by formulation
instead of quantizer, and return a real-valued latent rather than indices:
from medtokenizers import ContinuousTokenizer
model = ContinuousTokenizer(
dim=2, formulation="VAE", resolution=64, spatial_compression=8
).eval()
with torch.no_grad():
latent = model.tokenize(x) # (1, 4, 8, 8)
recon = model.detokenize(latent)
At evaluation time ContinuousTokenizer.encode returns the posterior mean, so
latents extracted with model.eval() are deterministic.
Set dim=3 for volumes; the same calls take (B, C, H, W, D) tensors.
Volumetric inference
A simulated T1-weighted brain volume ships inside the package (from the BrainWeb
Simulated Brain Database, with provenance in
src/medtokenizers/assets/README.md), so the
example runs without a download and works from an installed wheel.
example_volume_path() resolves it. Random initialisation is enough for a smoke
test; load your own weights for real reconstructions.
python examples/inference_on_brain.py --model-type maisi
Large volumes are reconstructed with Gaussian-weighted sliding windows:
import medrs
import torch
from medtokenizers import MAISITokenizer, example_volume_path
img = medrs.load(str(example_volume_path()))
volume = img.to_torch_with_dtype_and_device(dtype=torch.float32)
volume = volume.unsqueeze(0).unsqueeze(0) # (H, W, D) -> (1, 1, H, W, D)
volume = (volume - volume.min()) / (volume.max() - volume.min() + 1e-8)
model = MAISITokenizer().eval()
with torch.no_grad():
recon = model.reconstruct(volume, roi_size=(96, 96, 64), overlap=0.5)
Sliding-window reconstruction over a full 1 mm volume is memory-hungry. Reduce
roi_size or overlap if you are not on a GPU.
Tokenizers
| Class | Selected by | Output |
|---|---|---|
DiscreteTokenizer |
quantizer="VQ" | "FSQ" | "LFQ" | "RESFSQ" |
integer token grid |
ContinuousTokenizer |
formulation="VAE" | "AE" |
real-valued latent |
MAISITokenizer |
fixed MAISI configuration | real-valued latent |
TiTokTokenizer |
1D transformer tokenizer | fixed-length sequence of K tokens |
RAETokenizer (experimental) |
frozen foundation-model encoder | real-valued latent |
Adding a quantizer means subclassing BaseQuantizer (two abstract methods,
forward and indices_to_codes) and wiring it into DiscreteTokenizer. The
encoder-decoder and the tokenization I/O are unchanged by the choice.
TiTok
TiTok encodes an image or volume into a fixed-length 1D sequence of K tokens.
The encoder concatenates patch tokens with K learnable latent tokens and keeps
only the latter; the decoder reconstructs from quantized latents plus mask
tokens. Input shape must equal resolution, and each spatial dimension must be
divisible by patch_size.
from medtokenizers import TiTokTokenizer
tokenizer = TiTokTokenizer(
dim=2,
in_channels=1,
out_channels=1,
resolution=64,
patch_size=16,
num_tokens=32,
num_embeddings=1024,
hidden_dim=256,
)
indices, codes, quant_loss = tokenizer.encode(torch.randn(2, 1, 64, 64))
recon = tokenizer.decode(codes)
TiTok's original two-stage proxy-code warmup and decoder fine-tuning are not wired into the training scripts; implement that externally if you want the published recipe.
Evaluation
from medtokenizers import compute_lpips, compute_psnr, compute_ssim
psnr = compute_psnr(reference, reconstruction)
ssim = compute_ssim(reference, reconstruction)
lpips = compute_lpips(reference, reconstruction)
TokenizerEvaluator runs these over a loader, together with perplexity and
codebook utilisation for discrete models:
from medtokenizers import TokenizerEvaluator, load_tokenizer
model = load_tokenizer("./path/to/checkpoint")
results = TokenizerEvaluator(model, device="cuda").evaluate(test_loader)
No pretrained checkpoints ship with this repository.
Ecosystem
medtokenizers is one of three packages:
- medrs: medical-image I/O (NIfTI/DICOM) and preprocessing primitives.
- medtokenizers: tokenizers that compress scans into latents or token grids (this package).
- medlatents: generative models (autoregressive, MaskGIT, diffusion, flow matching, Bayesian flow) that learn over those tokens.
images -> medrs (load) -> medtokenizers (tokenize) -> medlatents (generate)
-> medtokenizers (detokenize) -> images
scripts/tokenize_dataset.py writes a directory of per-split .npz files plus
metadata.json, with key codes (int16) for discrete models and latents
(float16) for continuous ones.
Contributing
See CONTRIBUTING.md for development setup and the pull-request workflow, and CODE_OF_CONDUCT.md for community standards.
Citation
@inproceedings{chalcroft2026tokenizer,
title = {Tokenizer--Generator Coupling in Medical Image Generation},
author = {Chalcroft, Liam},
booktitle = {Advances in Neural Information Processing Systems},
year = {2026},
note = {arXiv:2608.07713}
}
@software{chalcroft_medtokenizers,
author = {Chalcroft, Liam},
title = {{medtokenizers}: Continuous and discrete tokenizers for volumetric medical imaging},
url = {https://github.com/liamchalcroft/medtokenizers},
version = {0.1.2}
}
Machine-readable metadata is in CITATION.cff.
License
MIT, see LICENSE.
Some modules derive from third-party projects: CompVis (Stable Diffusion,
taming-transformers) and lucidrains vector-quantize-pytorch under MIT, and
NVIDIA Cosmos-Tokenizer and MONAI/MAISI under Apache-2.0. Their terms are
preserved in THIRD_PARTY_NOTICES.md and
NOTICE. Two things are outside the MIT licence: NVIDIA MAISI
pretrained weights, if you use them, remain under NSCLv1; and the bundled
BrainWeb volume is data carrying its own citation requirement.
Metadata
Release files for medtokenizers 0.1.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| medtokenizers-0.1.2.tar.gz | 5.9 MB | Without provenance | Details for medtokenizers-0.1.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| medtokenizers-0.1.2-py3-none-any.whl | Python 3 | none | any | Details for medtokenizers-0.1.2-py3-none-any.whl |
Total release size: 11.7 MB
Release files / medtokenizers-0.1.2.tar.gz
| Download URL | medtokenizers-0.1.2.tar.gz |
|---|---|
| Size | 5.9 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
fb4825753deaa5f28fcd26ae19cc4385d8f82cc8584920a1c0717e65b435e3fa
|
|
BLAKE2b-256 checksum How to use checksums |
6847a7eecbdf36e3e3cab725e107c27ddb11ff6300e429d08362321745f2a634
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.11
|
Release files / medtokenizers-0.1.2-py3-none-any.whl
| Download URL | medtokenizers-0.1.2-py3-none-any.whl |
|---|---|
| Size | 5.8 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b58cf2e8b841d3d93ae0b5fc22d1ec800c95916001e1067ca6fb6561a5cfdcdf
|
|
BLAKE2b-256 checksum How to use checksums |
6e8ce7ac198ba314faaf9b1953849f1da53d41daf9b9b922260c52c75863666d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.11
|
Security for medtokenizers 0.1.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.