28 projects
tokenspeed-triton
A language and compiler for custom Deep Learning operations (vendor release for TokenSpeed)
tokenspeed-proton
A profiler for Triton (vendor release for TokenSpeed)
tokenspeed-smg
High-performance Rust-based inference gateway for large-scale LLM deployments
tokenspeed-smg-grpc-proto
SMG gRPC proto definitions for vLLM, TRT-LLM, MLX, TokenSpeed, and SGLang
tokenspeed-smg-grpc-servicer
SMG gRPC servicer implementations for LLM inference engines (vLLM, MLX, TokenSpeed, SGLang)
tokenrl
Placeholder for the upcoming TokenSpeed RL package
tokenspeed-scheduler
TokenSpeed C++ scheduler with Python bindings
tokenspeed-deepgemm
None
tokenspeed-cutedsl-kda
AOT CuteDSL KDA prefill kernels for TokenSpeed
tokenspeed-mla
Speed-of-light TokenSpeed MLA kernels for Blackwell SM100 and SM103.
tokenspeed-fa4
Flash Attention CUTE (CUDA Template Engine) implementation
tokenspeed-quack
A Quirky Assortment of CuTe Kernels
smg
High-performance Rust-based inference gateway for large-scale LLM deployments
tokenspeed-mooncake
A KVCache-centric Disaggregated Architecture for large-scale LLM inference and training.
tokenspeed-kernel-npu
Reserved placeholder for future TokenSpeed NPU kernel support.
tokenspeed-trtllm-kernel
Standalone TensorRT-LLM CUDA kernels as PyTorch custom ops
tokenspeed-fa3
FlashAttention-3
tokenspeed-flashkda
FlashKDA: Flash Kimi Delta Attention
tokenspeed-flashmla
None
tokenspeed-deepep
None
tokenspeed-fast-hadamard-transform
Fast Hadamard Transform in CUDA, with a PyTorch interface
tokenspeed
TokenSpeed is a speed-of-light LLM inference engine.
tokenspeed-kernel
None
tokenspeed-kernel-amd
TokenSpeed AMD-specific high-performance kernels.
tokenspeed-spdlog
TokenSpeed packaging for spdlog headers and CMake metadata.
tokenspeed-trie
A small harness for evaluating OpenAI-compatible inference endpoints with synthetic agentic workloads.
tokenspeed-iris
Triton-based framework for Remote Memory Access (RMA) operations with SHMEM-like APIs for multi-GPU programming.
tokenspeed-tritonblas
A Lightweight Triton-based BLAS Library
