multiscreen

MultiScreen

Implementation of Multiscreen proposed by Ken Nakanishi for Screening is Enough

Basically it is a non-softmax attention with ReLU squared activation, content similarity thresholding, and aggressive normalization of the values.

Install

$ pip install multiscreen

Usage

import torch
from multiscreen import MultiScreen

multi_screen = MultiScreen(
    num_tokens = 256,
    dim = 512,
    depth = 6,
    heads = 8,
    dim_keys = 16,     # paper says 16 or 32
    dim_values = 64    # paper says 64 or 128
)

token_ids = torch.randint(0, 256, (1, 1024))

logits = multi_screen(token_ids)
assert logits.shape == (1, 1024, 256)

Enwik8

$ uv run train.py

Citations

@misc{nakanishi2026screening,
    title   = {Screening Is Enough},
    author  = {Ken M. Nakanishi},
    year    = {2026},
    eprint  = {2604.01178},
    archivePrefix = {arXiv},
    primaryClass = {cs.LG},
    url     = {https://arxiv.org/abs/2604.01178},
}

@misc{zhang2021sparse,
    title   = {Sparse Attention with Linear Units},
    author  = {Biao Zhang and Ivan Titov and Rico Sennrich},
    year    = {2021},
    eprint  = {2104.07012},
    archivePrefix = {arXiv},
    primaryClass = {cs.CL}
}

@misc{horuz2025resurrectionrelu,
    title   = {The Resurrection of the ReLU},
    author  = {Coşku Can Horuz and Geoffrey Kasenbacher and Saya Higuchi and Sebastian Kairat and Jendrik Stoltz and Moritz Pesl and Bernhard A. Moser and Christoph Linse and Thomas Martinetz and Sebastian Otte},
    year    = {2025},
    eprint  = {2505.22074},
    archivePrefix = {arXiv},
    primaryClass = {cs.LG},
    url     = {https://arxiv.org/abs/2505.22074},
}

Metadata

Release files for multiscreen 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for multiscreen 0.2.0
File	Size	Uploaded
multiscreen-0.2.0.tar.gz	8.9 kB	May 13, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for multiscreen 0.2.0
File	Interpreter	ABI	Platform	Reset
multiscreen-0.2.0-py3-none-any.whl 8.1 kB May 13, 2026	Python 3	none	any	Details

Total release size: 17.0 kB

Release files / multiscreen-0.2.0.tar.gz

Download URL	multiscreen-0.2.0.tar.gz
Size	8.9 kB
Tags	Source
SHA-256 checksum How to use checksums	`0f3e9facb29922ac9daf48ee3344b92aab5eb749b805ad04fbc4707cea058f58`
BLAKE2b-256 checksum How to use checksums	`98e2e34b39172259161d1e9e292f7f09b3ed77a39054ae2bddacb17d587ad077`
Upload date	May 13, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.8.13`

Release files / multiscreen-0.2.0-py3-none-any.whl

Download URL	multiscreen-0.2.0-py3-none-any.whl
Size	8.1 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`5d9aa6fdf36a0aa02fa3a202d64cef049ec632842a052538c716486d8ccf7b35`
BLAKE2b-256 checksum How to use checksums	`3d7022331c250e2f7e5a1c1a9433c0c9999e4a8b9eb37476c7d7a9843106de96`
Upload date	May 13, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.8.13`