This release is a pre-release and may not be stable for production use.
ASpec
中文概览 · Documentation · Roadmap · Quickstart
Faster draft training. More room to experiment.
ASpec is a high-throughput training framework for speculative decoding draft models. It optimizes Draft forward, backward and optimizer execution, overlaps training with Target feature generation, and shares compatible Target features across independent experiments.
ASpec is developed from the TorchSpec codebase, building on its training infrastructure with further execution and workflow optimizations.
Use one workflow to explore algorithms and model structures, train at long context, compare independent experiments, and take a checkpoint through evaluation and export.
Why ASpec?
- Speed up Draft training itself. Optimized attention, fused kernels and selective head/loss computation reduce the work and memory traffic in each forward and backward pass.
- Keep the training pipeline moving. Overlap Target feature generation with Draft training, move features through a GPU tensor data path, and schedule training by useful work.
- One Target, multiple concurrent Draft experiments. Compare algorithms, structures or hyperparameters while generating compatible Target features once. Each Draft keeps its own weights, optimizer, progress and checkpoints, without a shared optimizer-step barrier. See shared Target training and the comparison and recovery walkthrough.
- Make long-context training practical. Combine sequence packing, context parallelism and optimized attention execution to reduce padding waste and distribute long sequences across GPUs.
- Explore without rebuilding the workflow. Switch between algorithm and model recipes with consistent configuration, reporting, recovery, evaluation and export.
Choose your Draft
Choose the algorithm and model structure for your workload. Draft and Target models can have different architectures:
| Dimension | Implemented choices |
|---|---|
| Draft algorithm | DSpark, DFlash, EAGLE3, EAGLE4, native MTP; experimental DFlash2 |
| FFN and routing | Dense, Qwen MoE, Ling MoE |
| Attention structure | MHA/GQA, materialized MLA, sliding-window attention (SWA), CSA, QSA, DSA |
| Execution | Sequence packing, context parallelism, shared Target features, static multi-node training |
See algorithm details and structure stages for the supported combinations, training phases and export formats. DSpark/DFlash with Qwen and Ling are the main optimization and recipe priorities; other implemented paths have explicit support boundaries, and DFlash2 remains experimental.
Scale from a first multi-GPU run to static multi-node training, with NVLink, TCP and RDMA data paths. Check supported paths and limitations, then browse the recipes for starting points.
Installation
In an activated Linux virtual environment:
pip install ling-aspec
This installs the Draft training environment. In a separate Target virtual environment, install the SGLang integration and apply its patch:
pip install ling-aspec-target
aspec patch-sglang
CUDA_VISIBLE_DEVICES=0 aspec target
Keep the Target service running, then start training in the Draft environment below.
The default connection is http://127.0.0.1:30000; set inference.target_service.address
for a remote Target. Each command uses its own environment.
Select the allocated GPUs separately for each role; this example uses GPU 0 for Target and GPU 1 for Draft.
See the installation guide for prerequisites, source installation
and release wheels, or use the container setup.
Optionally run aspec doctor to check the environment.
Quickstart
With two NVLink-connected GPUs:
CUDA_VISIBLE_DEVICES=1 aspec quickstart ling-tiny --output outputs/ling-tiny-001
# Or choose Qwen3-8B:
CUDA_VISIBLE_DEVICES=1 aspec quickstart qwen3-8b --output outputs/qwen3-8b-001
Both profiles use bundled raw conversations, their own tokenizer/template and an eight-step
train/resume/native-export check. Add --target /absolute/path/to/model to use a local checkpoint.
See Quickstart for the full workflow, then choose a
training recipe and prepare your data.
For training with Target-generated data and held-out evaluation, continue with the
Qwen3-8B DSpark recipe.
Additional workflows cover offline feature generation and replay and DSA dense → indexer → sparse training.
Performance
ASpec improves both Draft execution and the complete online training pipeline. The DSpark comparison measures Qwen3-8B, Qwen3.5-9B and Qwen3.8-27B on Hopper GPUs at short and long context. A representative Qwen3-8B / 4K result:
| Measurement | ASpec input tokens/s | / TorchSpec | / SpecForge | / AngelSpec |
|---|---|---|---|---|
| Draft-only, 1 GPU | 15,256.44 | 1.96× | 2.07× | 2.48× |
| Online steady window, 8 GPUs | 77,721.37 | 2.28× | 2.50× | 2.68× |
The comparison uses one original document per Draft rank and microstep (MBS1), aligned per-document anchor budgets and logical batch sizes, normal interconnects and sufficient buffers. Draft-only excludes Target execution, transport and feature loading; the online steady window includes them after warmup update eight. These are training input rates, not inference speedups or time to a given model quality. The full matrix reports software stacks, GPU splits and complete-loop rates separately.
A separate fixed-sequence Qwen3-8B / CP1 study compares training quality: BSHD validation CE stays close to TorchSpec; measured mean accepted length (tokens per verification round, including the Target bonus token) is 3.665717 for TorchSpec, 3.691665 for ASpec BSHD and 3.746300 for ASpec THD. With deterministic inference enabled, all three match Target-only greedy output token IDs on 256 questions across three repeats. These findings cover that training sequence and evaluation configuration.
For your own training run, see DSpark performance settings for cuDNN attention and direct-prefill options. Resource splits depend on the workload.
See benchmark results and methodology for measured workloads and reproducible comparisons. To assess the drafts you train, use the evaluation and export guide for held-out metrics, acceptance and downstream validation.
Learn more
- Configuration · Training and recovery
- Evaluation and export · Troubleshooting
- Contributing · Security
- Architecture · Project roadmap and status
- Supported paths · Contributor validation
Citation
If you find ASpec useful, please consider citing it:
@misc{aspec2026,
author = {Zhou, Aihui and Long, Lingkun and Yu, Chaofan and
Yang, Zhe and Peng, Cunyin and Zhang, Zhiqiang and Zhou, Jun},
title = {{ASpec}: A High-Throughput Training Framework for Speculative Decoding},
year = {2026},
url = {https://github.com/inclusionAI/ASpec}
}
Citation metadata is available in CITATION.cff.
License
ASpec is licensed under Apache 2.0. Third-party material retains its own licenses and notices; see the source headers and license texts. Model and dataset licenses are separate.
Acknowledgements
We thank TorchSpec, AngelSpec, and SpecForge for their contributions to the speculative decoding training ecosystem.
Metadata
Release files for ling-aspec-runtime 0.1.0b0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ling_aspec_runtime-0.1.0b0.tar.gz | 4.7 MB | Without provenance | Details for ling_aspec_runtime-0.1.0b0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ling_aspec_runtime-0.1.0b0-py3-none-any.whl | Python 3 | none | any | Details for ling_aspec_runtime-0.1.0b0-py3-none-any.whl |
Total release size: 5.7 MB
Release files / ling_aspec_runtime-0.1.0b0.tar.gz
| Download URL | ling_aspec_runtime-0.1.0b0.tar.gz |
|---|---|
| Size | 4.7 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5742d6b9a53bdc5a1c321512dd28352fa3fdc67c3c4f09d32ed732c97c96486f
|
|
BLAKE2b-256 checksum How to use checksums |
a5e3be19096e8e1395c4b9a078b47f2f368ebf687c90517e45868173b901cf03
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Release files / ling_aspec_runtime-0.1.0b0-py3-none-any.whl
| Download URL | ling_aspec_runtime-0.1.0b0-py3-none-any.whl |
|---|---|
| Size | 1.0 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
97d731da4219ada96375a549762c56a082cb8f6f881442075bcf04e1c3a69eed
|
|
BLAKE2b-256 checksum How to use checksums |
71b59ada5988bb46e695a8dbb370147f217671433e1eef532b6df765b797cf9d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Security for ling-aspec-runtime 0.1.0b0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.