Skip to main content

LLMC: Large Language Model Compressor

Stop burning money on LLM tokens.
Get 70-95% cost reduction through local RAG, intelligent routing, and containerized security.

Python License Tests


⚡ What is LLMC?

LLMC is a local-first RAG (Retrieval Augmented Generation) engine and intelligent router designed to drastically reduce the cost of using Large Language Models with your codebase.

Instead of sending your entire codebase to Claude or GPT-4, LLMC indexes your code locally, finds the exact relevant snippets (functions, classes, docs), and sends only what matters.

graph LR
    A[User Query] --> B(LLMC Router);
    B --> C{Local Index};
    C -->|Search| D[Relevant Context];
    D -->|Trim & Pack| E[Optimized Prompt];
    E --> F[LLM API];
    F --> G[Answer];
    style B fill:#f9f,stroke:#333,stroke-width:2px
    style E fill:#bfb,stroke:#333,stroke-width:2px

🚀 Quick Start

Get up and running in seconds.

1. Install

# One-line install
curl -sSL https://raw.githubusercontent.com/vmlinuzx/llmc/main/install.sh | bash

# Or via pip
pip install "git+https://github.com/vmlinuzx/llmc.git#egg=llmcwrapper[rag,tui,agent]"

2. Index Your Code

cd /path/to/your/project
llmc-cli repo register .

3. Save Money

# Search without using ANY tokens
llmc-cli search "authentication middleware"

# Launch the visual dashboard
llmc-cli tui

✨ Key Features

Feature Description
💸 Massive Savings Reduces token usage by 70-95% by sending only relevant context.
🔒 Security First New in v0.7.0: "Hybrid Mode" for trusted clients (host access) vs. Container Isolation for untrusted LLMs.
🧠 Polyglot RAG Smart parsing (TreeSitter) for Python, TS, JS, Go, Java, and technical docs.
🕸️ GraphRAG Understands your code structure (imports, calls, inheritance) to find related files automatically.
🖥️ TUI Dashboard Terminal UI to monitor indexing, search results, and costs. ⚠️ Work in progress — expect rough edges.
🔌 MCP Support Full Model Context Protocol server to integrate seamlessly with Claude Desktop.

🔍 Deep Dive

🛠️ Core RAG Engine
  • Local SQLite Index: Stores text + metadata without external dependencies.
  • Smart Embeddings: Caches embeddings to avoid re-computing unchanged files.
  • Context Trimmer: Packs the most relevant spans into a fixed token budget.
  • Enrichment: Uses small local models to tag and summarize code for better retrieval.
🛡️ Security & MCP
  • Hybrid Mode: Trusted clients get direct host access (~76% cheaper than docker overhead).
  • Container Isolation: Untrusted inputs run in Docker/nsjail.
  • Defense in Depth: Even if an LLM is "jailbroken" by prompt injection, it can't escape the container.
📊 Analytics & Routing
  • Intelligent Failover: Cascades from Local → Cheap Cloud → Premium Models.
  • Cost Tracking: Hard budget caps to prevent surprise bills.
  • Rate Limiting: Automatic token bucket throttling for API providers.

📚 Documentation

Full documentation is available in the DOCS/ directory:


📜 History

Originally created by David Carroll (the worst paragliding pilot in the TX Panhandle) after burning through his weekly API limits in days. This tool was born from the necessity to code more while spending less.


🤝 Contributing

We welcome PRs! Please check CONTRIBUTING.md before starting.

  1. Fork the repo
  2. Create your feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add some amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

Current Release: v0.9.1 "Back From Vacation"

Metadata

Release files for iflow-mcp_vmlinuzx-llmc 0.10.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for iflow-mcp_vmlinuzx-llmc 0.10.0
File Size Uploaded
iflow_mcp_vmlinuzx_llmc-0.10.0.tar.gz 889.4 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for iflow-mcp_vmlinuzx-llmc 0.10.0
File Interpreter ABI Platform
iflow_mcp_vmlinuzx_llmc-0.10.0-py3-none-any.whl Python 3 none any Details

Total release size: 1.7 MB

Release files / iflow_mcp_vmlinuzx_llmc-0.10.0.tar.gz

Download URL iflow_mcp_vmlinuzx_llmc-0.10.0.tar.gz
Size 889.4 kB
Tags Source
SHA-256 checksum
How to use checksums
af3db9629120f0b2ce9a6aad44feaed5372dc0381c2dfafe9bafb2f88d74c0cc
BLAKE2b-256 checksum
How to use checksums
02f12fbb8371a9b49ccf7304b7117a937bc3c944abde48f91633eda9eb424568
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.10.2 {"installer":{"name":"uv","version":"0.10.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Debian GNU/Linux","version":"13","id":"trixie","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / iflow_mcp_vmlinuzx_llmc-0.10.0-py3-none-any.whl

Download URL iflow_mcp_vmlinuzx_llmc-0.10.0-py3-none-any.whl
Size 766.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
65998a716c8d3f5853dbd7f9c5faf3ed2f2e8fd7fafc59f3a88fdc61468772ec
BLAKE2b-256 checksum
How to use checksums
d31da81cf81f806e7faf80718592abfab5b3217d5d76097536d7cbc7f59454df
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.10.2 {"installer":{"name":"uv","version":"0.10.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Debian GNU/Linux","version":"13","id":"trixie","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release history Release notifications | RSS feed

This release

0.10.0 This release

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page