SeqPacker

High-performance sequence packing for LLM training, written in Rust with Python bindings.

Documentation  |  Rust API  |  Benchmarks  |  Contributing

Training LLMs on variable-length sequences? Naive padding wastes 20-40% of GPU compute. SeqPacker packs sequences into fixed-size bins, achieving 95-99% utilization with 11 bin-packing algorithms — from O(n) streaming to near-optimal offline.

• 11 algorithms — NF, FF, BF, WF, FFD, BFD, FFS, MFFD, OBFD, OBFDP, HK
• Streaming API — bounded-space packing with incremental output
• HuggingFace integration — one-call pack_dataset for SFTTrainer / TRL
• PyTorch integration — GPU-ready tensors out of the box
• NumPy zero-copy — pass arrays directly, no conversion overhead
• Cross-platform — Linux, macOS, Windows; Python 3.9-3.13

Installation

# Python (pip)
pip install seqpacker

# Python (uv)
uv add seqpacker

# Rust
cargo add seqpacker

Quick Start

Python

from seqpacker import pack_sequences

lengths = [1000, 800, 600, 500, 400, 300, 200, 100]
result = pack_sequences(lengths, capacity=1024)

print(result.bins)        # [[0], [1, 7], [2, 4], [3, 5, 6]]
print(result.efficiency)  # 0.952...

Rust

use seqpacker::{Packer, PackStrategy};

let packer = Packer::new(1024)
    .with_strategy(PackStrategy::OptimizedBestFitDecreasing);

let result = packer.pack_lengths(&[1000, 800, 600, 500, 400, 300, 200, 100]).unwrap();
println!("Efficiency: {:.2}%", result.metrics.efficiency * 100.0);

Algorithms

11 bin-packing algorithms from O(n) online to optimal offline:

from seqpacker import Packer

# Use any algorithm by short name (default: obfd)
packer = Packer(capacity=2048, strategy="obfd")
result = packer.pack([500, 600, 400, 1000])

# List all available strategies
print(Packer.strategies())

Usage Modes

Batch Packing

Pack all sequences at once. Best for offline dataset preprocessing. All 11 algorithms available.

from seqpacker import Packer

packer = Packer(capacity=2048, strategy="obfd")
result = packer.pack(sequence_lengths)

for pack in result.packs:
    print(pack.sequence_ids, pack.lengths, pack.used)

print(f"Efficiency: {result.efficiency:.2%}")
print(f"Packs: {result.num_bins}")

Streaming

Feed sequences one at a time. Completed packs are emitted incrementally. Only bounded-space algorithms supported: NextFit (nf) and Harmonic-K (hk).

from seqpacker import StreamPacker

sp = StreamPacker(capacity=2048, strategy="nf")

for length in dataset_lengths:
    for pack in sp.add(length):
        process(pack)  # completed packs emitted as they fill

for pack in sp.finish():
    process(pack)      # flush remaining

Buffer + Batch

Accumulate sequences into a buffer and pack periodically. Requires no special library support -- just call pack() on each buffer. All algorithms available.

from seqpacker import Packer

packer = Packer(capacity=2048, strategy="obfd")
buffer = []

for sample in dataset_stream:
    buffer.append(len(sample["input_ids"]))
    if len(buffer) >= 10_000:
        result = packer.pack(buffer)
        for pack in result.packs:
            yield pack
        buffer.clear()

if buffer:
    result = packer.pack(buffer)
    for pack in result.packs:
        yield pack

Training Integration

HuggingFace Trainer

seqpacker.hf_utils builds a packed datasets.Dataset in one call -- ready for SFTTrainer, TRL, or any HF Trainer workflow. datasets is not a dependency -- import only when you need it.

from seqpacker.hf_utils import pack_dataset

tokenized = tokenizer(texts, truncation=True, max_length=2048)
ds = pack_dataset(tokenized["input_ids"], capacity=2048)

trainer = SFTTrainer(model=model, train_dataset=ds, ...)

The returned dataset includes input_ids, attention_mask, labels (shifted with boundary masking), and position_ids (per-sequence reset). See examples/sft_trainer.py for a complete fine-tuning script.

PyTorch DataLoader

seqpacker.torch_utils provides helpers for converting pack results into GPU-ready tensors. torch is not a dependency -- import only when you need it.

from seqpacker.torch_utils import packed_collate_fn
from torch.utils.data import DataLoader

collate = packed_collate_fn(capacity=2048, strategy="obfd")
loader = DataLoader(dataset, collate_fn=collate, batch_size=256)

for batch in loader:
    outputs = model(
        input_ids=batch.input_ids,
        position_ids=batch.position_ids,
        labels=batch.labels,
    )

Or convert a PackResult directly:

from seqpacker import pack_sequences
from seqpacker.torch_utils import pack_result_to_tensors

result = pack_sequences(lengths, capacity=2048)
batch = pack_result_to_tensors(result=result, token_ids=token_ids)
# batch.input_ids, batch.cu_seqlens, batch.position_ids, batch.labels, batch.attention_mask

See examples/pytorch_training.py for a complete training loop.

NumPy Support

Both list and NumPy array inputs are supported with zero-copy for NumPy:

import numpy as np
from seqpacker import Packer

packer = Packer(capacity=2048)
lengths = np.array([500, 600, 400, 1000], dtype=np.int64)
result = packer.pack(lengths)

# Flat NumPy output for maximum performance
items_flat, bin_offsets = packer.pack_flat(lengths)
bins = np.split(items_flat, bin_offsets)

Performance

SeqPacker achieves equal packing efficiency to competitors while being significantly faster:

Benchmarked on 10,000 sequences across real-world datasets (Alpaca, UltraChat, C4). See the interactive benchmark dashboard for detailed results.

Contributing

See CONTRIBUTING.md for setup instructions and development workflow.

make install       # Install dependencies
make build-dev     # Build the Rust extension
make test          # Run all tests (400 Rust + 249 Python)
make help          # See all commands

License

MIT