latentmesh 0.5.0

LangGraph-based multi-agent KV-cache communication protocol for LLMs

LatentMesh

Multi-agent Latent Space Communication for LLMs

Explore the docs »

Report Bug · Request Feature

Table of Contents

1. About The Project
2. Getting Started
   • Installation
3. Usage
4. Examples
5. Roadmap
6. License
7. Contact

About The Project

LatentMesh wires multiple LLM agents together in a LangGraph pipeline by strictly communicating via KV cache.

In standard architectures, when Agent B needs to read what Agent A computed, it must process exactly the same text history from scratch. With LatentMesh, Agent B can access the prior agent's KV cache states, cutting costs computation costs from $O(n^2)$ to $O(n)$. Our long term vision is to expand communication between any model regardless of differences in architecture.

(back to top)

Getting Started

Installation

Clone the repository and install in editable mode

git clone https://github.com/shayhacker/LatentMesh.git
cd LatentMesh
pip install -e .

Alternatively, install via PIP

not yet available

pip install latentmesh

Optional Dependencies:

• For persistent disk-backed caching (recommended for production):

  pip install latentmesh[disk]
  

• For booting a ready-to-go FastAPI endpoint:

  pip install latentmesh[server]
  

(back to top)

Usage

A basic multi-agent configuration building a Plan -> Reason -> Review text pipeline.

from langgraph.graph import StateGraph, START, END
from latentmesh import LatentLLM, LatentState
from latentmesh.primitives import PlanPrimitive, ReasonPrimitive, ReviewPrimitive
from latentmesh.persistent_cache import MemoryKVStore, GlobalPrefixCache

# 1. Provide the cache to skip re-encoding string prefixes
store = MemoryKVStore()
cache = GlobalPrefixCache(store)

# 2. Use a standard HF model
llm = LatentLLM("Qwen/Qwen3-0.6B", device="cuda", global_cache=cache)

# 3. Create discrete agent primitives
planner  = PlanPrimitive(llm)
reasoner = ReasonPrimitive(llm)
reviewer = ReviewPrimitive(llm)

# 4. Connect via LangGraph 
builder = StateGraph(LatentState)
builder.add_node("planner", planner)
builder.add_node("reasoner", reasoner)
builder.add_node("reviewer", reviewer)

builder.add_edge(START, "planner")
builder.add_edge("planner", "reasoner")
builder.add_edge("reasoner", "reviewer")
builder.add_edge("reviewer", END)

graph = builder.compile()

# 5. Run it forward sequentially
result = graph.invoke({
    "messages": [{"role": "user", "content": "What is the cosine of 45 degrees?"}],
    "tokens_so_far": 0,
})

print(result["latent"].text)
print(f"Total tokens generated: {result['tokens_so_far']}")

(back to top)

Examples

For more involved architectures like consensus routing and dynamic state inspection, check out examples/:

Example	Description
sequential.py	Plan → Reason → Review zero-shot pipeline
complex.py	Fast consensus parallel-voting with VotingPrimitive
hierarchical.py	Dynamic routing based on model confidence scoring

(back to top)

Roadmap

• Implement HuggingFace model backend wrapping
• Add global prefix matching via pygtrie
• Create Memory and Disk KV Stores
• Multi-GPU parallelization for VotingPrimitive
• Implement LRU/TTL eviction policies for the global prefix cache
• Publish v1.0.0 to PyPI

See the open issues for a full list of proposed features (and known issues).

(back to top)

License

Distributed under the MIT License. See LICENSE for more information.

(back to top)

Contact

• Yash Ranjith - [e-mail]
• Hiroki Kimiwada - [e-mail]
• William Peng - [e-mail]
• Atharva Rao - [e-mail]

Project Link: https://github.com/shayhacker/LatentMesh

(back to top)