llm-layer-collector

Load and dispatch individual transformer layers from HuggingFace model checkpoints, rather than instantiating a whole model at once. Given a model directory (config + safetensors shards), it can materialize just the embedding, a range of decoder layers, the final norm, or the LM head — each as a standalone torch.nn.Module — and run computation through them.

This powers the layer-sharding used by language-pipes for distributed inference, but has no dependency on it and can be used on its own.

Install

pip install llm-layer-collector

Usage

from llm_layer_collector import LlmLayerCollector

collector = LlmLayerCollector(model_dir="/path/to/model", cache_file="cache.json")
embedding = collector.load_input_embedding()
layers = collector.load_layer_set(0, 4)   # decoder layers 0..4 (end inclusive)
norm = collector.load_norm()
head = collector.load_head()

Supported architectures

Llama, Phi-3, Qwen3, Qwen3-MoE, Gemma3, Gemma4, Ministral3. See src/llm_layer_collector/modeling/ for per-architecture support.

License

MIT

Development Status
- 4 - Beta
Intended Audience
- Science/Research
License
- OSI Approved :: MIT License
Operating System
- OS Independent
Programming Language
- Python :: 3.10
Topic
- Scientific/Engineering :: Artificial Intelligence
Typing
- Typed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

llm_layer_collector-1.1.0.tar.gz (33.0 kB view details)

Uploaded Jul 24, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

llm_layer_collector-1.1.0-py3-none-any.whl (24.3 kB view details)

Uploaded Jul 24, 2026 Python 3

File details

Details for the file llm_layer_collector-1.1.0.tar.gz.

File metadata

Download URL: llm_layer_collector-1.1.0.tar.gz
Upload date: Jul 24, 2026
Size: 33.0 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for llm_layer_collector-1.1.0.tar.gz
Algorithm	Hash digest
SHA256	`00123fb1e60f8ab9f89232a60bfa64f670895ba832ec03929d188fe34f694a4b`
MD5	`b0b2357bbc5289ca4394e0f3a61c818f`
BLAKE2b-256	`f15793225752050df73903295472b782d76e166d9ea34abe39869ece8d401655`

See more details on using hashes here.

Provenance

The following attestation bundles were made for llm_layer_collector-1.1.0.tar.gz:

Publisher: publish-packages.yml on erinclemmer/language-pipes

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: llm_layer_collector-1.1.0.tar.gz
- Subject digest: 00123fb1e60f8ab9f89232a60bfa64f670895ba832ec03929d188fe34f694a4b
- Sigstore transparency entry: 2230313084
- Sigstore integration time: Jul 24, 2026
Source repository:
- Permalink: erinclemmer/language-pipes@290435434278489898dd3735ff7c8c958038b1df
- Branch / Tag: refs/heads/main
- Owner: https://github.com/erinclemmer
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish-packages.yml@290435434278489898dd3735ff7c8c958038b1df
- Trigger Event: push