slmify 0.2.0

Point it at your LLM logs. See exactly where you're overpaying — fine-tune a small model, cache, or downgrade.

slmify

Point it at your LLM logs. See exactly where you're overpaying — and what to fix: fine-tune a small model, cache, or downgrade.

LLM workload rightsizing, as a local-first CLI. slmify ingests your production LLM logs or agent traces, groups them into workloads (workflow step → prompt template → semantic cluster), scores each workload's SLM-fit on a banded two-axis system (feasibility × dollar opportunity), computes honest, range-based savings with break-even math, recommends the cheapest safe action per workload — and exports training-ready datasets plus configs for whichever fine-tuning stack you already use.

NVIDIA's research estimates 40–70% of LLM calls in agentic systems are SLM-replaceable — but sketched the conversion pipeline without implementing it. slmify measures that number on your traffic.

Status: v0.2 (pre-release). Analyzer, zero-training probe, HTML report, dataset exports, and the local trainer are all in.

Principles (non-negotiable)

• Local-first, zero telemetry, zero required API keys. Your prompts never leave your machine. The default embedding model is a 30 MB static model; the tool runs offline.
• No false precision, ever. Bands not decimals; ranges not points; every assumption printed. Expect frequent, honest KEEP recommendations — a tool that tells you not to fine-tune 7 of 10 clusters is one you can trust.
• Evidence over heuristics. Every score comes with the exact metrics behind it; every formula is documented and user-overridable via --policies.
• Neutral. Reads every vendor's logs, exports to every trainer. No login, no gates, no nags.

Two zero-friction first runs

uvx slmify              # no arguments: analyzes your local Claude Code agent
                        # traces (~/.claude/projects) — a personal verdict from
                        # data already on your laptop

uvx slmify demo         # bundled synthetic corpus + precomputed embeddings:
                        # a full report in ~30s, zero network, works on a plane

Then the real thing:

uvx slmify analyze ./logs.jsonl --html report.html   # format auto-detected
uvx slmify analyze ./logs.csv --map "prompt=question,response=answer,model=model_name"

# prove feasibility before spending anything: zero training, k-shot, your own endpoint
slmify probe --clusters top3 --endpoint http://localhost:11434/v1 --model qwen3:4b-instruct

slmify export c07        # train/eval JSONL + Unsloth/Axolotl/TRL/OpenAI/vLLM-router stubs
slmify train c07         # [train] extra: LoRA on the same recipe (CUDA/Unsloth)
slmify eval c07 --endpoint http://localhost:8000/v1 --model tuned --baseline-model base
slmify pricing --update  # refresh the price snapshot (only network call you didn't configure)

Supported formats: Claude Code traces (~/.claude/projects), Codex CLI (~/.codex/sessions) and Gemini CLI (~/.gemini/tmp) sessions, Langfuse observation exports, OpenAI-style JSONL (fine-tuning chat format, chat-completion objects, request/response pairs), and generic JSONL/CSV with dotted-path --map — which covers most homegrown logging tables and gateway DB exports. Running LiteLLM? slmify.integrations.litellm_logger.SlmifyLogger is a drop-in callback that logs complete request/response pairs straight into slmify's native format (SpendLogs don't store content). OTel GenAI lands in v0.3. Adapters are pluggable via the slmify.adapters entry-point group.

What the report gives you

For every discovered workload:

• SLM-fit band (A "Prime" → D "Poor fit") from 10 published, documented metrics — output contraction, schema stability, task-type priors, template mass, semantic cohesion, reasoning depth, and more — with the top-3 evidence bullets and a confidence tag. Raw numbers live in analysis.json, not the headline.
• Probe evidence on demand: slmify probe runs a candidate SLM against 30 held-out samples per cluster (k-shot, zero training) through any OpenAI-compatible endpoint and adjusts the band ±1 step. Graders run cheapest-first (exact → JSON-schema → embedding similarity); an LLM judge only runs if you supply a judge endpoint, and flat judge scores automatically switch to ranking mode.
• Dedicated-GPU break-even math: the report tells you the sustained calls/hour where a dedicated L4/A10G/A100 beats your current spend — and says "use serverless" when you're below it.
• Measured spend and savings ranges — {low, point, high} with the assumptions listed, priced from a vendored LiteLLM pricing snapshot, monthly volumes extrapolated with a bootstrap CI over your observed days. A 3-day log honestly yields wide bars.
• An action stack: FINE_TUNE_SLM, DOWNGRADE_TRIAL, EXACT_CACHE, SEMANTIC_CACHE, PROMPT_CACHE / TRIM, FIX_QUALITY_FIRST (with a named remedy), or an honest KEEP — decided by auditable ordered rules you can override.
• Cache economics as floor / realistic / ceiling — exact-duplicate rate is the zero-risk floor, semantic near-duplicate mass the ceiling.
• Multi-turn conversations in flat logs are reconstructed into episodes (conservative prefix matching), so volumes, duplicate rates, and dataset exports don't double-count growing message arrays.

slmify export <cluster> writes train.jsonl / eval.jsonl (OpenAI chat format — accepted by OpenAI FT, Unsloth, Axolotl, and TRL) plus ready-to-run config stubs. The eval set is a stratified holdout that is never included in training data, so you can verify the fine-tune on untainted samples.

How slmify compares

	slmify	OpenAI Distillation	Braintrust / LangSmith	Kiln
Finds what to fix in your traffic	✅ scored workloads	—	clusters, no scoring/savings	task-first
Cross-vendor (reads any logs)	✅	OpenAI only	their platform	n/a
Runs locally, data never leaves	✅	—	—	✅
Proves SLM feasibility pre-training	✅ probe	—	—	—
$ savings with break-even math	✅ ranges	—	—	—
Exports to any trainer	✅	—	partial	✅

As of 2026-07. These tools solve adjacent problems — several pair well with slmify rather than replacing it.

Install

uv tool install slmify        # or: pipx install slmify / pip install slmify

Core install is light: polars, numpy, scikit-learn, model2vec, drain3, pydantic, typer, rich, httpx, pyyaml. No torch, no database, no compiled clustering libs. Extras: [st] (sentence-transformers embedders), [train] (local LoRA fine-tuning, v0.2).

Offline degradation ladder: demo needs zero network → the first real analysis downloads the 30 MB default embedder once (then cached) → fully offline, use --embedder hash (TF-IDF, already installed) — degraded clusters, still useful, never a dead end.

Privacy posture

Report exemplars are masked by default (<EMAIL>, <NUM>, <ID>, …) via the same template miner that names your clusters; --show-raw disables. Dataset exports contain raw text (training needs real data) and say so loudly. There is no telemetry of any kind.

Benchmarks

benchmarks/ holds reproducible runs: a ground-truth validation of the whole grouping pipeline on the bundled labeled corpus (purity 1.00, V-measure 0.94), and a one-command public-data benchmark (WildChat-1M / LMSYS-Chat-1M) that measures the SLM-able share of real traffic against NVIDIA's 40–70% agentic estimate.

Roadmap

• v0.3 — the standard: --watch/CI diff mode, OTel GenAI adapter, in-process adapter eval, an open calibration study on public data, plugin authoring docs, --backend mlx (Apple Silicon training).

License

Apache-2.0.