axm-smelt 0.2.0

Deterministic token compaction for LLM inputs

axm-smelt — Deterministic token compaction for LLM inputs

---

axm-smelt reduces token consumption for LLM inputs by applying deterministic compaction strategies — whitespace collapsing, structural transforms, and optional lossy simplifications. It works as a CLI, Python API, and MCP tool for AI agents.

📖 Full documentation

Features

• Format detection — auto-detect JSON, YAML, XML, TOML, CSV, Markdown, and plain text
• Token counting — exact counts via tiktoken (o200k_base), with len//4 fallback
• 10 strategies — minify, drop_nulls, flatten, tabular, round_numbers, strip_quotes, dedup_values_with_refs, collapse_whitespace, compact_tables, strip_html_comments
• Composable pipeline — chain strategies explicitly or use presets (safe, moderate, aggressive)
• CLI — axm-smelt compact|check|count|version with --preset/--strategies/--file/--output flags
• MCP tool — SmeltTool for use by AI agents via axm-mcp
• Modern Python — 3.12+ with strict typing

Installation

uv add axm-smelt

Quick Start

CLI

# Compact from stdin (uses safe preset by default)
echo '{"name": "Alice", "age": 30}' | axm-smelt compact

# Compact a file with the aggressive preset
axm-smelt compact --file data.json --preset aggressive

# Compact with specific strategies, write output to file
axm-smelt compact --file data.json --strategies minify,drop_nulls --output out.json

# Analyze token waste without modifying (shows per-strategy estimates)
axm-smelt check --file data.json

# Count tokens
echo 'hello world' | axm-smelt count

Python API

from axm_smelt import smelt, check, count

# Compact using the safe preset (default)
report = smelt('{\n  "name": "Alice",\n  "age": 30\n}')
print(f"{report.savings_pct:.1f}% saved")
# Tokens: 14 -> 9

# Compact with explicit strategies
report = smelt(data, strategies=["minify", "drop_nulls"])

# Compact with a preset
report = smelt(data, preset="aggressive")

# Analyze without transforming
report = check('{"data": [1, 2, 3]}')
for strat, pct in report.strategy_estimates.items():
    print(f"  {strat}: {pct:.1f}%")

# Count tokens
tokens = count("hello world")

MCP (AI Agent)

axm-smelt is available as an MCP tool via axm-mcp. AI agents can call smelt_compact(data, preset="moderate") directly.

See the MCP how-to guide for details.

CLI Commands

Command	Description
axm-smelt compact	Read from stdin/file, output compacted text; savings reported to stderr
axm-smelt check	Analyze token waste without transforming; shows strategies with positive savings
axm-smelt count	Print token count
axm-smelt version	Print version string

All commands accept --file PATH to read from a file instead of stdin. compact additionally accepts --strategies LIST, --preset NAME, and --output PATH.

Strategies

Name	Category	Description
minify	whitespace	Compact JSON/YAML/XML whitespace (parse + re-serialize; preserves data)
drop_nulls	structural	Recursively remove None, "", [], {} values
flatten	structural	Collapse single-child wrapper dicts ({"a":{"b":1}} → {"a.b":1})
tabular	structural	Convert list[dict] JSON to pipe-separated tables
round_numbers	cosmetic	Round floats to N decimal places (default: 2)
strip_quotes	cosmetic	Remove quotes on simple alphanumeric JSON keys
dedup_values_with_refs	structural	Replace repeated long strings (≥20 chars, ≥2 occurrences) with aliases. Output is wrapped in a {_refs, _data} envelope — not format-preserving.
collapse_whitespace	whitespace	Collapse consecutive blank lines and strip trailing whitespace on prose/Markdown (skips structured formats and fenced code blocks)
compact_tables	whitespace	Remove padding whitespace from Markdown table cells (skips fenced code blocks)
strip_html_comments	cosmetic	Remove <!-- … --> HTML comments from Markdown/plain text (skips fenced code blocks)

Presets

Preset	Strategies	Use when
safe	minify, collapse_whitespace	Data-preserving — keeps the parsed value identical for structured formats (JSON/YAML/…) and never touches fenced code. Not byte/whitespace-lossless on prose/Markdown: collapse_whitespace collapses blank-line runs and strips trailing whitespace.
moderate	minify, drop_nulls, flatten, dedup_values_with_refs, tabular, strip_quotes, collapse_whitespace, compact_tables, strip_html_comments	Structural transforms are acceptable
aggressive	minify, drop_nulls, flatten, tabular, round_numbers, dedup_values_with_refs, strip_quotes, collapse_whitespace, compact_tables, strip_html_comments	Maximum savings, may alter float precision

Development

This package is part of the axm-forge workspace.

git clone https://github.com/axm-protocols/axm-forge.git
cd axm-forge
uv sync --all-groups
uv run --package axm-smelt --directory packages/axm-smelt pytest -x -q

Migration notes

• 0.1.0.dev0 → next: the dedup_values strategy was renamed to dedup_values_with_refs to signal that its output is wrapped in a {_refs, _data} envelope and is not format-preserving. The old name is removed from the registry (no alias) — pipelines referencing "dedup_values" will raise KeyError. Update CLI invocations (--strategies dedup_values_with_refs) and library calls (Pipeline(strategies=["dedup_values_with_refs"])) accordingly.

License

Apache-2.0 — © 2026 axm-protocols