TokenSpeed is a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability. Our goal is to be the most performant inference engine for production agentic workloads.
Core components:
- Modeling layer: local-SPMD design with a static compiler that generates collective communication from module-boundary placement annotations, so users do not hand-write parallelism logic.
- Scheduler: C++ control plane and Python execution plane. Request lifecycle, KV cache ownership, and overlap timing are encoded as a finite-state machine, with safe KV resource reuse enforced by the type system at compile time.
- Kernels: pluggable, layered kernel system with a portable public API and a centralized registry including one of the fastest MLA (Multi-head Latent Attention) implementations on Blackwell for agentic workload.
- Entrypoint: SMG-integrated AsyncLLM for low-overhead CPU-side request handling.
News
- [2026/07] TML Inkling at Day 0: FP4 Inference on NVIDIA and AMD with TokenSpeed. [blog]
- [2026/06] Deep dive into the design and optimization of TokenSpeed-Kernel. [blog]
- [2026/05] 🚀 TokenSpeed hits 580 TPS on Qwen3.5-397B-A17B for agentic workloads. [blog]
- [2026/05] TokenSpeed announced — a speed-of-light LLM inference engine for agentic workloads. [blog]
Blogs and Talks
For technical blogs, conference talks, and engineering articles from LightSeek Foundation, visit the LightSeek Blog.
Performance Comparison
Documentation
Start here:
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file tokenspeed-0.1.0.tar.gz.
File metadata
- Download URL: tokenspeed-0.1.0.tar.gz
- Upload date:
- Size: 1.1 MB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
92512d9b2b71289dbf7e5cf40815744394f241778ae2837da3229e5a4bec9ed8
|
|
| MD5 |
dd921a87d1ef1bdea5e3095352a264e5
|
|
| BLAKE2b-256 |
a651f4ef5987816a41f89a64462442f7dc9e01eb4becbbdb06bdbf18b4ccf4e4
|
Provenance
The following attestation bundles were made for tokenspeed-0.1.0.tar.gz:
Publisher:
release-pypi.yml on lightseekorg/tokenspeed
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
tokenspeed-0.1.0.tar.gz -
Subject digest:
92512d9b2b71289dbf7e5cf40815744394f241778ae2837da3229e5a4bec9ed8 - Sigstore transparency entry: 2230262579
- Sigstore integration time:
-
Permalink:
lightseekorg/tokenspeed@c53cc0fbdea9bde0a80e9f4cddb0c8e26656e9c9 -
Branch / Tag:
refs/heads/bot/pypi-trusted-publishing - Owner: https://github.com/lightseekorg
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release-pypi.yml@c53cc0fbdea9bde0a80e9f4cddb0c8e26656e9c9 -
Trigger Event:
workflow_dispatch
-
Statement type:
File details
Details for the file tokenspeed-0.1.0-py3-none-any.whl.
File metadata
- Download URL: tokenspeed-0.1.0-py3-none-any.whl
- Upload date:
- Size: 1.4 MB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2649dfcb853bbba74a065fa10d3280513e5755da057b6a9cbf6b82ff47d70b2a
|
|
| MD5 |
0285a972df27a454e8e8e3a3279a7f15
|
|
| BLAKE2b-256 |
9a4fa142b34cbd8fa5709f1b8037a8489f86d0df282b8d29bf0d435a156bb775
|
Provenance
The following attestation bundles were made for tokenspeed-0.1.0-py3-none-any.whl:
Publisher:
release-pypi.yml on lightseekorg/tokenspeed
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
tokenspeed-0.1.0-py3-none-any.whl -
Subject digest:
2649dfcb853bbba74a065fa10d3280513e5755da057b6a9cbf6b82ff47d70b2a - Sigstore transparency entry: 2230263024
- Sigstore integration time:
-
Permalink:
lightseekorg/tokenspeed@c53cc0fbdea9bde0a80e9f4cddb0c8e26656e9c9 -
Branch / Tag:
refs/heads/bot/pypi-trusted-publishing - Owner: https://github.com/lightseekorg
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release-pypi.yml@c53cc0fbdea9bde0a80e9f4cddb0c8e26656e9c9 -
Trigger Event:
workflow_dispatch
-
Statement type:
