LATTICE

LLM Transport & Efficiency Layer
Make every LLM call cheaper, faster, and safe — without changing your model.

LATTICE sits between your application and any LLM provider. It compresses prompts, caches responses, manages concurrency (TACC), supports a native binary protocol, and routes coding agents through one self-hosted proxy. Your app sends standard OpenAI-format requests; LATTICE makes them smaller, faster, and cache-friendlier.

It is not a router. LATTICE never changes your model, never falls back between providers, never guesses. One provider per request. LATTICE optimises transport and execution.

Table of Contents

• Installation
• Quick Start
• Architecture
• Novel Technology
• Compression Pipeline
• Safety
• Observability
• Supported Providers
• CLI Reference
• Agent Integration
• Development
• Migrating from v0.x
• Documentation
• License

Installation

pip install lattice-transport

Optional extras:

pip install "lattice-transport[redis]"   # Multi-process session store
pip install "lattice-transport[mcp]"     # MCP tool support
pip install "lattice-transport[all]"     # Everything

Requirements: Python 3.10+. No external services required for single-process mode.

Quick Start

# Start the proxy
lattice proxy run --port 8787

# Point any OpenAI SDK at it
export OPENAI_BASE_URL=http://localhost:8787/v1

# Or route an agent through it
lattice lace claude

from lattice import LatticeClient

client = LatticeClient()
response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Explain transport protocols"}],
)
print(response.choices[0].message.content)

Every request is automatically compressed, cached, and optimized in proxy mode — zero application code changes.

Architecture

Application
   │ OpenAI / Anthropic API format
   ▼
LATTICE Proxy :8787
   │
   ├── state/     Session, segments, SemanticCache
   ├── planner/   RequestClassifier → UnifiedPlanner → ExecutionPlan
   ├── pipeline/  Pipeline.compress() — IR-native transforms + gates
   ├── telemetry/ Metrics, downgrade, cost, agent stats
   └── providers/ adapters/ (17) + transport/ (HTTP pool, TACC, streaming)
            │
            ▼
       LLM Provider (exactly one per request)

Request flow

1. Client sends POST /v1/chat/completions (or Anthropic /v1/messages).
2. SessionManager creates or loads a session (CAS versioning).
3. content_profiler builds a semantic profile; UnifiedPlanner produces an ExecutionPlan.
4. Pipeline.compress() runs transforms with policy, budget, risk, and MILV gates.
5. SemanticCache checks exact hash, then approximate fingerprint.
6. On miss, the provider adapter sends via HTTP/2 pool; TACC manages admission.
7. Response reverse pass + x-lattice-* headers via LatticeHeaderMiddleware.

→ Runtime architecture

Novel Technology

LATTICE applies classical systems techniques to LLM workloads — transport and execution, not model features.

Batching overhead reduction and long-conversation dedup savings are measured in the canonical benchmark suite — see Claim traceability ^{1 2}.

Compression Pipeline

LATTICE ships 20 transforms (list_transform_names() in lattice.transforms.registry). Six run in the default pipeline; three are execution-only (batching, speculative, delta); the rest are planner-selected or off by default. Every transform is safety-classified and risk-gated.

exec = execution-only transform (outside default pipeline list).

Headline compression on the canonical feature suite (ollama-cloud / kimi-k2.6:cloud): 40.3% average reduction ³. Pipeline latency ~36 ms ⁴.

→ Transform reference · Claim traceability · Feature parity checklist (61 rows)

Safety

Transforms are classified SAFE, CONDITIONAL, or DANGEROUS. A 0–100 semantic risk score gates lossy transforms; expansion guards cap token growth.

→ Safety guide · SIG · RATS · PSG · MILV

Observability

curl http://localhost:8787/stats | jq
curl http://localhost:8787/metrics

• /stats — transforms, sessions, pools, TACC, maintenance, downgrades
• /metrics — Prometheus counters and histograms
• Response headers — x-lattice-compression, x-lattice-session-id, x-lattice-delta, x-lattice-cost-usd, x-lattice-provider, x-lattice-transforms-applied

→ Observability

Supported Providers

17 direct adapters. No routing — one provider per request.

→ Provider details

CLI Reference

lattice proxy run --port 8787
lattice proxy start|stop|restart|status
lattice init
lattice lace|unlace <agent>
lattice info|config|status|health|doctor
lattice benchmark --suite feature   # wraps benchmarks/evals/cli.py

→ CLI reference

Agent Integration

lattice lace claude    # Claude Code
lattice lace codex     # OpenAI Codex
lattice lace cursor    # Cursor
lattice lace opencode  # OpenCode
lattice lace copilot   # GitHub Copilot

lattice doctor (no args) checks all five agents. lattice init applies durable config; lattice lace uses transient routing + tunnel sidecar.

→ Integrations

Development

git clone https://github.com/Harsh-Daga/lattice
cd lattice
uv sync
uv run pytest tests/ -q              # 2016 collected, 1801 passed (215 skipped)
uv run pytest tests/contract/ -q
uv run ruff check src/ tests/ benchmarks/
uv run ruff format --check src/ tests/ benchmarks/
uv run mypy src/lattice/

uv run python benchmarks/evals/cli.py --suite all \
  --providers ollama-cloud \
  --provider-model ollama-cloud=kimi-k2.6:cloud \
  --iterations 3 --warmup 1

→ AGENTS.md for AI agent contributors

Migrating from v0.x

Internal Python imports changed in 1.0.0. CLI and HTTP are stable.

→ Migration guide · CHANGELOG

Documentation

→ Full index

Claim footnotes

License

MIT © Harsh Daga

GitHub · Issues · PyPI · Changelog