🤑 economize

Composable efficiency techniques for Hugging Face language models

economize makes text generation with Hugging Face causal language models faster and cheaper. It implements more than 50 published efficiency techniques, from speculative decoding and early exit to pruning, quantization and KV-cache compression, and attaches any combination of them to an unmodified transformers model with a single function call.

• One interface for many methods. Drafters such as EAGLE-3 and Medusa, dynamic-depth methods, weight compression and KV-cache compression share the same configuration and decoding API.
• Free combination. Techniques can be stacked, for example 2:4 pruning, int4 quantization, activation sparsity and a drafter in one model. economize attaches them in a well-defined order.
• The base model stays intact. The model keeps its modules and its forward method, so it continues to work with generate, PEFT, the Hugging Face Hub and the rest of the ecosystem.
• Fast, lossless decoding. Batches of prompts of different lengths, chains and draft trees, lossless speculative sampling, CUDA graphs and speculative speculative decoding over several GPUs.
• Training and serving. Drafters and other added components train with a built-in loop or with transformers.Trainer, and export to vLLM and SGLang.
• Extensible. New techniques, model families, verifiers and backends are added by registering a class, without changing the library.

Installation

pip install economize

economize requires Python 3.10 or later, PyTorch 2.4 or later and transformers 5.0 or later. The optional integrations (PEFT, quantization kernels, training, evaluation, vLLM, SGLang) are extras, listed in Getting started.

Quick start

A published EAGLE-3 head attached to its base model generates exactly the tokens of the base model, only faster:

import torch
import economize
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct", dtype=torch.bfloat16,
                                            device_map="cuda")
model = economize.load(base, "yuhuili/EAGLE3-LLaMA3.1-Instruct-8B")  # a Hub repository or a local folder

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
ids = tokenizer("The water cycle describes", return_tensors="pt").input_ids.cuda()
out = model.generate(ids, max_new_tokens=128)
print(tokenizer.decode(out[0]))

Several techniques combine in one call. Techniques that need calibration data receive it through calibration:

from economize.techniques.eagle3 import Eagle3Config
from economize.techniques.prune import PruneConfig
from economize.techniques.quant import QuantConfig
from economize.techniques.sparsity import TealConfig

model = economize.apply(
    base,
    PruneConfig(method="wanda", sparsity="2:4"),                # 2:4 sparse weights
    QuantConfig(format="int4", method="gptq", group_size=128),  # int4 weights
    TealConfig(sparsity=0.4),                                   # activation sparsity
    Eagle3Config(),                                             # a drafter, trained with economize.train.fit
    calibration=calibration_ids,  # a LongTensor [N, S], an iterable of batches or a callable
)

The documentation covers installation, decoding options, training, serving, a tutorial for every technique and how to extend the library.

Supported techniques

The techniques work on 46 model types, including dense, mixture-of-experts and hybrid attention/Mamba models. Techniques lists every technique with its registered name, its backends and the techniques it combines with.

Model zoo

Drafters trained with economize are published on the Hugging Face Hub in the collection economize drafters, for example:

The speedups are lossless, at batch size 1 on one H100, averaged over chat, code, math, summarization and long-context prompts. The model zoo lists every checkpoint with its speedup per task group, and Benchmarks compares economize with the published speedups of other checkpoints.

Limitations

Version 0.4 does not yet support dynamic draft trees or speculative speculative decoding on models with recurrent layers, trainable hook-based splits on tensor-parallel models (frozen ones are supported), or speculative speculative decoding on tensor-parallel models. Some techniques cannot yet be exported to vLLM, in which case the export raises an error that states the reason.

Development

pip install -e ".[dev]"
pytest              # unit, technique, integration and oracle tests on the CPU
pytest -m gpu       # batched decoding on a CUDA device

Contributing describes the checks and the design rules for contributions, and Extending how to add a technique, a model family or a backend.

Citation

If you use economize in your research, please cite it:

@software{laitenberger2026economize,
  author  = {Laitenberger, Filipe},
  title   = {economize: Composable Efficiency Techniques for Hugging Face Language Models},
  year    = {2026},
  url     = {https://github.com/Thiggel/economize},
  version = {0.4.0}
}

Please also cite the papers of the techniques you use.

License

economize is released under the Apache License 2.0. See LICENSE for details.