_ ____ _
| | | __`\(_)
| | __ _ ___ ___ _ _ __ _ __ _ ___ | |__) | |_ __ ___ ___
| | / _` |/ _ \ / _ `| | | |/ _` |/ _` |/ _ \ | ___/| | '_ \ / _ \/ __|
| |___| (_| | | | | (_| | |_| | (_| | (_| | __/ | | | | |_) | __/\__ \
|______\__,_|_| |_|\__, |\__,_|\__,_|\__, |\___| |_| |_| .__/ \___||___/
__/ | __/ | | |
|___/ |___/ |_|
Peer-to-peer distributed inference for open-source language models
Documentation & site: https://languagepipes.com
Language Pipes is an open-source distributed inference system built on the transformers library that splits large language model computation across multiple machines. By separating the model's text-handling components (embedding and output head) from its intermediate transformer layers, Language Pipes enables peer-to-peer inference.
Features
- OpenAI-compatible API
- Automatic model download by HuggingFace ID
- Interactive TUI for configuration, monitoring, and control
- Decentralized peer-to-peer network with optional AES encryption
How It Works
Language models process input through a sequence of transformer layers. Each layer performs matrix multiplications between learned weights and a hidden state tensor, passing the result to the next layer. Language Pipes distributes these layers across machines, splitting the memory cost across the network while keeping the text-handling components on the origin node.
The architecture provides architectural separation: layer models operate on continuous-valued tensors rather than discrete text while the end models keep text data on trusted systems. The privacy documentation provides a probabilistic threat model that quantifies the difficulty of known inversion attacks under various mitigation configurations.
Further reading:
- Architecture Overview: runtime components and inference flow
- Job Processor State Machine: how jobs traverse the distributed pipeline
Installation
Requires Python 3.10+. A GPU is optional — each node can run on CPU or CUDA, and a node only needs enough memory to hold the layers it hosts (memory scales with how many layers you assign it). For GPU support, install the appropriate PyTorch version for your CUDA configuration:
https://pytorch.org/get-started/locally/
See System requirements and network requirements for details, including what happens when a node drops mid-generation.
Install from pip:
pip install language-pipes
Quick Start
Launch the interactive TUI:
language-pipes
From the main menu, select New Configuration and give it a name to create a TOML config and open the dashboard (or Load Configuration to reopen one you've created before).
The dashboard is organized into tabs along the top: Home, Network, Models, Pipes, and Jobs. A fresh configuration has no node ID yet, so the only option on Home is Configure Network Server. Set a Node ID under Network > Configure, then return to Home and select Start Network Server. Once the network is running, the dashboard exposes the rest of setup: load models under Models > Layer Models / End Models, and configure and start the OpenAI-compatible API under Jobs > Server.
Configuration can also be edited directly as TOML files and run headlessly. See the CLI reference for details on running a saved configuration from the command line.
Two Node Example
This example distributes Qwen/Qwen3-1.7B across two computers. Node 1 hosts the End Model, so prompts and responses stay on Node 1, plus enough layers to fit in its memory. Node 2 hosts the remaining layers.
Node 1 (First Computer)
language-pipes
Select New Configuration and name it (e.g. node-1).
- Network > Configure: set Node ID to
node-1and ensure Network IP is set to this machine's local IP address. Leave Network Key empty to disable encryption for this example. Peer Port defaults to5000. - Back on Home, select Start Network Server.
- Models > Installed: select Install New Model and enter
Qwen/Qwen3-1.7Bto download it. - Models > Layer Models: select Add Layer Model, choose
Qwen/Qwen3-1.7B, a device (cpuorcuda:0), and a memory budget in GB (e.g.2), then Save Model. Confirm to load it now. - Models > End Models: select Add End Model, choose
Qwen/Qwen3-1.7B, and confirm to load it now. - Jobs > Server: ensure the Port is set to
8000and select Start Server.
Node 2 (Second Computer)
language-pipes
Select New Configuration and name it (e.g. node-2).
- Network > Configure: set Node ID to
node-2. Under Bootstrap Nodes, add an entry with node-1's IP address and peer port (5000) so this node joins node-1's network. - Back on Home, select Start Network Server.
- Models > Installed: install
Qwen/Qwen3-1.7Bas on Node 1. - Models > Layer Models: add
Qwen/Qwen3-1.7Bwith a device and memory budget covering the remaining layers (e.g.2oncpu).
Once both nodes have loaded their layers, Pipes > Complete shows a completed pipe for Qwen/Qwen3-1.7B, and the model is ready for inference via node-1's Job Port.
Test the API
The model is accessible via the OpenAI-compatible API.
Example using the OpenAI Python library:
from openai import OpenAI
client = OpenAI(
base_url="http://127.0.0.1:8000/v1", # node-1 IP address and Job Port
api_key="not-needed" # only required if api_keys is set in the config
)
response = client.chat.completions.create(
model="Qwen/Qwen3-1.7B",
max_completion_tokens=100,
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Write a haiku about distributed systems."}
]
)
print(response.choices[0].message.content)
Install the OpenAI library with: pip install openai
See the OpenAI-compatible API documentation for the full endpoint reference and sampling parameter descriptions.
Supported Models
Language Pipes supports Qwen3 (dense and MoE), Phi, Meta Llama 2/3, Gemma 3, Gemma 4, and Ministral 3. See the supported-models list for a more complete list.
Contributing
Contributions are welcome! Bug reports, documentation fixes, and especially new model architectures. See CONTRIBUTING.md for dev setup, how to run the tests, and PR expectations, and the Code of Conduct. Questions and ideas are welcome in GitHub Discussions.
Packages
Language Pipes is built on two components that are also published as standalone PyPI packages, maintained from this repo under packages/:
- llm-layer-collector — load individual transformer components (embedding, decoder layers, norm, head) from sharded HuggingFace checkpoints and run per-architecture computation. Docs
- distributed-state-network — an encrypted peer-to-peer state-sharing network over HTTP. Docs
Language Pipes pins exact versions of both, so pip install language-pipes always pulls the matching releases — you never have to update them yourself.
Dependencies
Documentation
The docs are published as a website at https://erinclemmer.github.io/language-pipes (built from this folder by website/). The Markdown source of truth lives in documentation/:
1 maintainer
Metadata
1 maintainer
Release files for language-pipes 2.5.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| language_pipes-2.5.0.tar.gz | 391.9 kB | With provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| language_pipes-2.5.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 549.3 kB
Release files / language_pipes-2.5.0.tar.gz
| Download URL | language_pipes-2.5.0.tar.gz |
|---|---|
| Size | 391.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
43b44925c8c67744362d6e8425dec3515216d4fc13704143b1a6d889aaac20f4
|
|
BLAKE2b-256 checksum How to use checksums |
b714d3fb9b848982639cc4412c537e6fcd5414beb83fe7751b4ac14558ab9984
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 9, 2026.
Transparency logRelease files / language_pipes-2.5.0-py3-none-any.whl
| Download URL | language_pipes-2.5.0-py3-none-any.whl |
|---|---|
| Size | 157.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
aa8f8f03e0ebb0665ae016055a414e86859ad696c908d31333f528428da0c795
|
|
BLAKE2b-256 checksum How to use checksums |
fd374b5924cbee5c4438a5eefa9a6ef69c7c6c8c708f0af566f3512e51ef6060
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 9, 2026.
Transparency logSecurity for language-pipes 2.5.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 2 files in this release have been uploaded with provenance metadata.
Attestation for language_pipes-2.5.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 9, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
