This release is a pre-release and may not be stable for production use.
Stackformers
Composable PyTorch transformers for research.
Build vision, spatial, and sequence models with interchangeable attention, positional encodings, normalization, and feedforward components. Start with a preset, then replace individual components as your experiments evolve.
- Experiment with architecture: explicit, serializable configurations and structural protocols let you supply your own components.
- Work with variable-length data: run the same encoder weights on padded or packed token batches, with explicit token coordinates.
- Connect research and deployment: examples cover image patches, point clouds, causal modeling, custom components, and cached ONNX Runtime decoding.
Try the current API
Python 3.11 or newer is required. This checkout is on the 5.0 release-candidate series. To run the examples shown here against the matching source:
git clone https://github.com/Red-Eyed/stackformers.git
cd stackformers
uv run examples/padded_and_packed.py
uv run prepares the project environment automatically. For a published release in an
existing project:
uv add stackformers
An unpinned install may resolve a different stable release whose API differs from this checkout. Use the documentation matching your installed version. For experiments, pin the exact version or commit you validated; see the research guide.
Run your first encoder
import torch
import stackformers as sf
model = sf.TransformerEncoder(
sf.plain_encoder_config(dim=64, heads=1, num_layers=2, ff_mult=3.0)
).eval()
sequences = [torch.randn(8, 64), torch.randn(5, 64)]
padded = sf.PaddedTensor.from_list(sequences)
with torch.no_grad():
padded_output = model(padded) # (2, 8, 64)
packed_output = model(padded.to_packed()) # (13, 64)
The two calls share model weights. Packed output contains valid tokens only; padded output retains batch and sequence axes. See the layout parity example for a numerical comparison. Packed input does not guarantee faster execution: backend, hardware, dtype, and length distribution matter.
Choose an example
Examples run on CPU without dataset or model downloads. From a repository checkout,
uv run prepares the project environment automatically.
| Your task | Start here | Run |
|---|---|---|
| Image patches and 2-D positions | Image encoder | uv run examples/image_encoder.py |
| Continuous spatial coordinates | Point-cloud encoder | uv run examples/point_cloud_encoder.py |
| Unequal sequence lengths | Padded and packed | uv run examples/padded_and_packed.py |
| Replace a model component | Custom encoders | uv run examples/subclassed_encoder.py |
| Autoregressive token modeling | Causal language model | uv run examples/causal_language_model.py |
| Export an encoder–decoder | ONNX Runtime guide | uv run examples/encoder_decoder_onnxruntime.py |
These are integration examples with synthetic inputs, not trained models or task-accuracy benchmarks. Stackformers provides transformer components; your application owns tokenization, data processing, training, and task heads.
What can you change?
| Component | Included choices |
|---|---|
| Attention | Global self/cross attention, local self-attention, GQA/MQA |
| Token layout | Padded or packed, with explicit positions |
| Positional encoding | RoPE-1D, RoPE-2D, continuous-coordinate RoPE-ND, or none |
| Feedforward | SwiGLU, HardSwishGLU, GEGLU, GELU, ReLU² |
| Normalization | RMSNorm, LayerNorm; pre, post, sandwich, or reordered placement |
| Model assembly | Encoders, decoders, cross-attenders, variable-width encoders |
| Deployment | Projection packing, compile/export paths, cached decoder adapters |
Presets build components from discriminated Pydantic configurations. For deeper changes, wire modules directly or implement an owning module's protocol. The configuration and API guide covers both approaches.
Compatibility and limits
- Global attention uses PyTorch SDPA. Eligible CUDA fp16/bf16 packed workloads use PyTorch's variable-length attention; other paths can fall back to padded execution.
- Compile and export support depend on the component and input path. Eager list/nested adapters and deployment cache strategies have separate contracts.
- Cached deployment supports specified built-in classes; custom forward overrides need their own deployment integration.
- No speedup or downstream accuracy improvement is claimed without a matching benchmark.
See compatibility evidence and inference contracts before selecting a deployment path.
Documentation
- Examples and expected behavior
- Research reproducibility and citation
- API, layouts, configuration, and migration
- Deployment and inference
- Release history
- Contributing
- Adoption and evidence roadmap
Development
git clone https://github.com/Red-Eyed/stackformers.git
cd stackformers
just sync
just check
The Justfile owns formatting, linting, type-checking, tests, and builds.
Citation and license
If Stackformers contributes to your research implementation, cite the software using CITATION.cff and report the exact version or commit used. No associated research paper or conference endorsement is implied.
Released under the MIT license.
Metadata
Release files for stackformers 5.0.0rc22
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| stackformers-5.0.0rc22.tar.gz | 589.9 kB | Without provenance | Details for stackformers-5.0.0rc22.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| stackformers-5.0.0rc22-py3-none-any.whl | Python 3 | none | any | Details for stackformers-5.0.0rc22-py3-none-any.whl |
Total release size: 730.2 kB
Release files / stackformers-5.0.0rc22.tar.gz
| Download URL | stackformers-5.0.0rc22.tar.gz |
|---|---|
| Size | 589.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f6d0b75a983899120f94517a753a8bf8d84b0822b468b024e06e552ec8dc4402
|
|
BLAKE2b-256 checksum How to use checksums |
da11ba448138f2cc8b8a9166b591798cc7941494f2936b274d4c38979ff245d3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.1 {"installer":{"name":"uv","version":"0.12.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / stackformers-5.0.0rc22-py3-none-any.whl
| Download URL | stackformers-5.0.0rc22-py3-none-any.whl |
|---|---|
| Size | 140.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
349ad420ff2d4150bcbc70fa7dbd081cca315aa44624a3a40601b7c02bb804a8
|
|
BLAKE2b-256 checksum How to use checksums |
8647ce0ee83e80c20c02a2878bf89386c633b0055522a0463962a2a8f3ea20f6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.1 {"installer":{"name":"uv","version":"0.12.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for stackformers 5.0.0rc22
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.