axm-smelt 0.3.1

axm-smelt — Deterministic token compaction for LLM inputs

axm-smelt reduces token consumption for LLM inputs with deterministic transformations. It detects input formats, tries ordered strategies, and keeps candidates that reduce tokens or shorten text at equal token count.

Features

• Detect JSON, YAML, XML, TOML, CSV, Markdown and plain text.
• Compact inputs with ordered strategies and configurable presets.
• Analyze independent strategy savings without replacing the input text.
• Count tokens and report the counter backend through Python and AXM tools.

Installation

Requires Python 3.12+. In a uv project:

uv add axm-smelt

Quick Start

After installation, run this example with uv run python:

import json
from axm_smelt import check, count, smelt

data = '{\n  "name": "Alice",\n  "age": 30\n}'
report = smelt(data)
assert json.loads(report.compacted) == json.loads(data)
print(report.compacted, report.savings_pct)

analysis = check(data)
print(analysis.strategy_estimates)  # Independent estimates; do not add them
assert analysis.savings_pct == report.savings_pct
print(count("hello world"))

This prints compacted JSON, savings, independent strategy estimates and a count for hello world. The assertions verify this example's JSON values and the agreement between the default pipeline and its analysis.

The Python surface also exports SmeltReport, Format, CounterBackend and __version__. Compaction returns new text; it does not overwrite a file.

Usage

AXM tools and CLI

Three axm.tools registrations supply CLI, MCP and DAG access:

printf '{"name": "Alice", "notes": null}\n' | uv run axm smelt --json-output
uv run axm smelt --data '{"a": 1, "b": null}' --strategies '["minify", "drop_nulls"]'
uv run axm smelt_count --data 'hello world' --model o200k_base

Use --input-path ./payload.json to read a UTF-8 file. Nonempty data takes precedence over the file, then non-interactive stdin. JSON-looking --data is decoded by the CLI; use a file/stdin to preserve its original whitespace baseline. Plain stdout includes a header; --json-output prints the data mapping. See CLI reference for errors and encoding.

For MCP, install the package into the server environment and call axm_call(name="smelt", arguments={...}) in façade mode. Integration guide.

Strategies and fidelity

The name safe is not a general losslessness guarantee. XML significant whitespace, YAML inline comments and Markdown layout can change. Structural transforms can lose values/types and produce output that is not valid JSON. Inspect strategy behavior and limits before choosing a preset. More strategies do not guarantee greater savings.

Detected formats: JSON, YAML, XML, TOML, CSV, Markdown and text. TOML/CSV have no dedicated compactor. Counts use tiktoken; Claude and unknown model names use an o200k_base proxy, not the target model's exact tokenizer or billing. The smelt/check pipelines always use o200k_base.

Documentation

• Published documentation
• Getting started
• Compact data
• Python/tool contracts
• CLI reference
• Strategy behavior and fidelity

The MkDocs home page is docs/index.md; it is separate from this README.

Development

This package belongs to the axm-forge workspace.

git clone https://github.com/axm-protocols/axm-forge.git
cd axm-forge
uv sync --all-packages --all-groups
uv run --package axm-smelt --directory packages/axm-smelt pytest -x -q

License

Licensed under Apache-2.0. See LICENSE.