local-llm-wrapper 26.5

Local-first LLM wrapper (import local_llm_wrapper) with pluggable transports (Ollama, Apple, optional Claude Code CLI) and a stable text-in text-out API.

local_llm_wrapper

Local-first LLM wrapper with a stable text-in text-out API. Supports pluggable transports (Ollama, Apple, and an optional Claude Code CLI cloud transport) with structured output helpers and retry for unattended runs.

Package name: local-llm-wrapper (import as local_llm_wrapper).

Overview

• Library-first wrapper for text models with a small, stable API.
• Local-first design; an optional cloud transport (Claude Code CLI) is also available.
• Fallback across transports when a backend is unavailable.
• Structured helpers for rename/sort workflows with strict parsing and format-fix retries.

Features

• Pluggable transports in local_llm_wrapper/transports/.
• Fallback chain in LLMClient with guardrail and context-window handling.
• Structured prompts and parsers for rename, keep, and sort workflows.
• Prompt sanitization and output normalization utilities.
• Quiet mode for non-verbose runs.
• Unified API for text prompts and chat-style message lists.

Quick start

import local_llm_wrapper.llm as llm

client = llm.LLMClient(
	transports=[
		llm.AppleTransport(),
		llm.OllamaTransport(model=llm.choose_model(None)),
	],
	quiet=True,
)

response = client.generate("Say hello in one sentence.", max_tokens=120)
print(response)

Chat example

import local_llm_wrapper.llm as llm

client = llm.LLMClient(
	transports=[llm.OllamaTransport(model="qwen3.5:9b-q4_K_M")],
	quiet=True,
)
messages = [
	{"role": "system", "content": "Answer in one sentence."},
	{"role": "user", "content": "What is a mutex?"},
]
print(client.generate(messages=messages, max_tokens=120))

CLI example

source source_me.sh && python3 llm_generate.py -p "Say hello in one sentence." -t 80

CLI usage

llm_generate.py is a repo-root helper for quick prompt tests against the Ollama transport.

source source_me.sh && python3 llm_generate.py --help

Options:

• -p, --prompt: Prompt text to send (default: "Say hello in one sentence.").
• -m, --model: Override the auto-selected model (default: auto).
• -t, --max-tokens: Maximum tokens to generate (default: 80).
• -q, --quiet: Suppress progress output (default).
• -v, --verbose: Show progress output.

CLI chat demo

llm_chat.py is an interactive chat loop that uses chat-style messages.

source source_me.sh && python3 llm_chat.py

Options:

• -m, --model: Override the auto-selected model (default: auto).
• -s, --system: Optional system message to start the chat.
• -t, --max-tokens: Maximum tokens to generate per response (default: 240).
• -q, --quiet: Suppress progress output (default).
• -v, --verbose: Show progress output.

XML tag demo

llm_xml_demo.py requests a tagged response and extracts <answer> from the model output.

source source_me.sh && python3 llm_xml_demo.py -p "Say hello in one sentence."

Structured helpers

The engine includes structured helpers for common file-organization tasks.

import local_llm_wrapper.llm as llm

client = llm.LLMClient(transports=[llm.OllamaTransport(model="qwen3.5:9b-q4_K_M")])
item = {
	"path": "notes.txt",
	"name": "notes",
	"ext": "txt",
	"description": "meeting notes",
}
result = client.sort([item])
print(result.assignments)

Install extras

The Apple backend requires at least one Apple SDK. Install via optional extras:

pip install local-llm-wrapper[apple]           # prebuilt wheels, no Xcode needed (recommended)
pip install local-llm-wrapper[apple-official]   # Apple's official SDK, requires Xcode 26+
pip install local-llm-wrapper[apple-all]        # both backends (for dev/CI)

When both SDKs are installed, the transport prefers apple-fm-sdk when it is available and safe to use, and falls back to apple-foundation-models otherwise.

Claude Code transport

ClaudeCodeTransport delegates to the claude CLI in non-interactive --print mode.

import local_llm_wrapper.llm as llm

client = llm.LLMClient(
	transports=[
		llm.AppleTransport(),
		llm.OllamaTransport(model=llm.choose_model(None)),
		llm.ClaudeCodeTransport(model="sonnet"),
	],
	quiet=True,
)

response = client.generate("Say hello in one sentence.", max_tokens=120)
print(response)

Caveats when using ClaudeCodeTransport:

• Prompts are sent to the Anthropic cloud, not processed locally or offline.
• The CLI runs in --print mode which skips the workspace trust dialog; only use it from directories you trust.
• By default, all Claude Code tools are disabled (--tools ""), so the transport does not intentionally grant workspace read, write, or execute access.

Transports

• Apple: local_llm_wrapper/transports/apple.py (dual-backend: apple-fm-sdk and apple-foundation-models).
• Ollama: local_llm_wrapper/transports/ollama.py.
• Claude Code CLI: local_llm_wrapper/transports/claude_code.py.
• Protocol: local_llm_wrapper/transports/base.py.

Errors

Standardized exception types live in local_llm_wrapper/errors.py so callers can handle guardrails, context window errors, and transport availability consistently.

Testing

source source_me.sh && python3 -m pytest tests/

Docs

• Usage: docs/USAGE.md
• Architecture: docs/CODE_ARCHITECTURE.md
• Facade reference: docs/LLM_FACADE.md
• API migration guide: docs/API_IMPLEMENTATION_GUIDE.md
• File layout: docs/FILE_STRUCTURE.md
• Changelog: docs/CHANGELOG.md

Notes

• This repo is text-only; image or OCR workflows belong elsewhere.
• Configuration is explicit through arguments or config objects, not custom environment variables.