llmkit

Run any LLM — local or via API — with one config file. No framework lock-in. Works on Windows, Mac, Linux.

pip install llmkit-cli

llmkit run "explain this codebase in one sentence"

Switch between OpenAI, Anthropic, Groq, Mistral, DeepSeek, Together, and local Ollama models by changing one line in llm.yaml. No code changes. No redeploy.

Install

Recommended (PyPI):

pip install llmkit-cli

Or clone and run the platform installer:

# Linux
bash install.sh

# Mac
bash install.mac.sh

# Windows (PowerShell as admin)
./install.ps1

VS Code extension: install vscode-llmkit to run prompts and commands from the editor status bar.

Configure

Create llm.yaml in your project (or run llmkit init for a guided setup):

provider: groq          # local | openai | anthropic | groq | together | deepseek | mistral
model: llama-3.3-70b-versatile

For API providers, add your key to .env:

GROQ_API_KEY=your_key_here

Commands

# One-shot prompt
llmkit run "explain this codebase in one sentence"

# Validate config and check provider reachability
llmkit check

# Interactive wizard to create llm.yaml
llmkit init

# Pin current model to llm.lock (commit this file)
llmkit lock

# Manage API keys in .env
llmkit env

# Generate a commit message from staged changes
llmkit commit

# Generate a PR title + body vs main/master
llmkit pr

# Review staged/unstaged diff for bugs
llmkit review

# Run a coding agent on a task
llmkit agent "refactor this module to use dataclasses"

# Plan only — no tools executed, just a numbered plan
llmkit agent --plan "add pagination to the API"

# Approve mode — confirm each shell command before it runs
llmkit agent --approve "run the test suite and fix any failures"

Switch providers

Change one line in llm.yaml, run llmkit check, done:

# was: provider: groq
provider: anthropic
model: claude-3-5-haiku-20241022

No code changes. No redeploy. Works in CI too.

Fallback

If the primary provider fails, llmkit cascades to the next:

provider: groq
model: llama-3.3-70b-versatile
fallback:
  - provider: openai
    model: gpt-4o-mini
  - provider: anthropic
    model: claude-3-5-haiku-20241022

Budget

Cap token usage per run:

budget:
  max_tokens_per_run: 10000
  warn_at: 8000

API providers

Local models (via Ollama)

provider: local
model: llama4       # or qwen3, deepseek-r1, mistral, phi4, gemma3

llm.lock

Run llmkit lock to pin your model runtime. Commit llm.lock alongside your code — same idea as package-lock.json:

locked_at: "2026-06-26T12:00:00Z"
provider: groq
model: llama-3.3-70b-versatile

5-minute team setup

# 1 — install
pip install llmkit-cli

# 2 — set one API key (Groq free tier works)
echo "GROQ_API_KEY=your_key_here" > .env

# 3 — create config
llmkit init

# 4 — verify
llmkit check

# 5 — run
llmkit run "explain this codebase in one sentence"

No server. No IDE plugin required. No code to write.

Examples

python examples/chat.py
python examples/stream.py
python examples/tools.py        # function calling
python examples/multiround.py   # multi-round conversation
python examples/embed.py        # embeddings + cosine similarity
python examples/agent.py        # coding agent
python examples/mcp_agent.py    # MCP agent

node examples/chat.js           # JS — all providers
node examples/stream.js         # JS — streaming

Links

• PyPI: https://pypi.org/project/llmkit-cli/
• GitHub: https://github.com/abinz-aiml/llmkit