json2toon
Convert JSON structures into TOON (Token-Oriented Object Notation) and back again, with tooling to measure token savings and create ready-to-send prompts for LLMs.
At a glance
- JSON <-> TOON conversion (
json_to_toon,toon_to_json,convert_file) - Configurable formatting (indent, separators, quoting, table layouts)
- Automatic table layout for uniform object arrays
- Token counting and comparison reports
- Prompt helpers to wrap TOON payloads for LLMs
- Rich CLI with syntax highlighting and reports
- Fully covered by pytest (39 passing tests)
Install
pip install json2toon
CLI usage
The CLI is exposed as json2toon.
# JSON -> TOON
json2toon to-toon input.json -o output.toon
# TOON -> JSON (pretty by default)
json2toon to-json input.toon -o output.json
# Token comparison report
json2toon report input.json -f table
json2toon report input.json -f json # machine-readable
Flags of interest:
-c/--configload a JSON config file (see Config section)-p/--prettypretty-print TOON or JSON when writing to stdout-P/--no-prettydisable pretty JSON output-o/--outputwrite to a file instead of stdout
Examples:
# Convert and view with highlighting
json2toon to-toon sample.json --pretty
# Round trip via files
json2toon to-toon sample.json -o sample.toon
json2toon to-json sample.toon -o sample.roundtrip.json
# Get a JSON report of token savings
json2toon report sample.json -f json > stats.json
Python API
Import from json2toon:
from json2toon import (
json_to_toon, toon_to_json, convert_file, get_conversion_stats,
ToonEncoder, ToonDecoder, ToonConfig,
compare_formats, count_tokens, generate_report,
create_llm_prompt, create_response_template, wrap_in_code_fence, add_system_prompt,
)
Core helpers (core.py)
json_to_toon(data, config=None) -> str: Encode any JSON-serializable Python data to TOON.toon_to_json(toon_str, config=None) -> Any: Decode TOON back to Python data.convert_file(input_path, output_path, to_toon=True, config=None): File-level conversion in either direction.get_conversion_stats(data, config=None, output_format="text") -> dict: Compute token counts, savings, and a formatted report (text/json/markdown).
Encoder (encoder.py)
ToonEncoder.encode(data) -> str: Main entry. Handles primitives, objects, arrays, tables, indentation, string quoting/escaping, and inline vs block arrays.- Table layout: for uniform arrays of dicts (based on
uniformity_thresholdandmin_table_rows), arrays render as ASCII tables.
Decoder (decoder.py)
ToonDecoder.decode(toon_str) -> Any: Parses tables, list items, objects, primitives, inline JSON, and escaped strings/newlines.
Config (config.py)
ToonConfigfields:separator(default:)table_separator(default|),header_separatormax_inline_array_length,compress_primitive_arraysmax_string_length,quote_stringsindent_size,max_nesting_depthuniformity_threshold,min_table_rows
get_default_config()returns defaults.save_config(config, path),load_config(path)to persist/restore.
Analysis (analyzer.py)
analyze_structure(data, config) -> StructureInfo: Classifies primitives, objects, arrays, and detects uniform arrays.is_uniform_array(arr, threshold=0.8) -> (bool, keys)used by the encoder.should_use_table_format(data, config) -> boolselects table layout.
Metrics (metrics.py)
count_tokens(text, encoding_name="cl100k_base") -> int: Usestiktokento count tokens.compare_formats(data, encoder, encoding_name="cl100k_base") -> ComparisonResult: Token counts for JSON vs TOON.generate_report(comparison, output_format="text") -> str: Formats text/json/markdown reports.
Prompt helpers (prompt.py)
create_llm_prompt(toon_str, system_prompt=None): Wrap TOON payloads for model input.create_response_template(): Skeleton for expected model replies.wrap_in_code_fence(text, language="toon"): Add triple-backtick fences.add_system_prompt(prompt, system_prompt): Prepend system guidance.
CLI (cli.py)
- Commands:
to-toon,to-json,report. All support optional config loading and pretty-print toggles. Success output uses ASCII-only markers for Windows compatibility.
Exceptions (exceptions.py)
EncodingError, DecodingError, AnalysisError, ConfigurationError all derive from Json2ToonError.
Usage examples
Programmatic round trip
from json2toon import json_to_toon, toon_to_json
data = {"name": "Alice", "scores": [95, 87, 92]}
toon = json_to_toon(data)
restored = toon_to_json(toon)
assert restored == data
Custom config
from json2toon import ToonConfig, ToonEncoder
config = ToonConfig(indent_size=4, quote_strings=True, table_separator='|')
encoder = ToonEncoder(config)
toon = encoder.encode({"users": [{"id": 1, "name": "Bob"}, {"id": 2, "name": "Eve"}]})
print(toon)
Token savings report
from json2toon import ToonEncoder, compare_formats, generate_report
data = {"items": [
{"id": 1, "name": "Alpha"},
{"id": 2, "name": "Beta"},
]}
encoder = ToonEncoder()
comparison = compare_formats(data, encoder)
print(generate_report(comparison, output_format="markdown"))
Build an LLM prompt
from json2toon import json_to_toon, create_llm_prompt
data = {"request": "Summarize", "payload": {"text": "Hello world"}}
toon = json_to_toon(data)
prompt = create_llm_prompt(toon, system_prompt="You are a concise assistant.")
print(prompt)
Configuration file example
Save as toon_config.json:
{
"separator": ":",
"table_separator": "|",
"header_separator": "-",
"max_inline_array_length": 8,
"compress_primitive_arrays": true,
"quote_strings": true,
"indent_size": 2,
"uniformity_threshold": 0.75,
"min_table_rows": 2
}
Use it:
json2toon to-toon data.json -c toon_config.json -o data.toon
Project layout
src/json2toon/
analyzer.py # structure detection, uniformity checks
cli.py # Typer-based CLI
config.py # ToonConfig dataclass + load/save
core.py # top-level helpers and file conversion
decoder.py # TOON -> Python
encoder.py # Python -> TOON
exceptions.py # custom exception types
metrics.py # token counting and reports
prompt.py # LLM prompt utilities
tests/ # pytest suite (39 tests)
Development
python -m pytest # run tests
python -m pytest --cov # run with coverage
The suite exercises all public APIs (encode/decode, configs, CLI, metrics, prompt helpers) and passes on Windows with ASCII-only CLI output.
Metadata
Release files for py-json2toon 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| py_json2toon-0.1.0.tar.gz | 21.4 kB | Without provenance | Details for py_json2toon-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| py_json2toon-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for py_json2toon-0.1.0-py3-none-any.whl |
Total release size: 38.7 kB
Release files / py_json2toon-0.1.0.tar.gz
| Download URL | py_json2toon-0.1.0.tar.gz |
|---|---|
| Size | 21.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0497b367bc2b902c82ab99665b90cd066b504f876cb7e2fd71ab18afd6553f5b
|
|
BLAKE2b-256 checksum How to use checksums |
506c7835e97aa943efd2f605744fb84217353141bae12754a47c39583f671bef
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.5
|
Release files / py_json2toon-0.1.0-py3-none-any.whl
| Download URL | py_json2toon-0.1.0-py3-none-any.whl |
|---|---|
| Size | 17.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b3ea1319bac0fa051564f4d92ef883d6e1d93c4c66062b3e3eddbd7cec7acde2
|
|
BLAKE2b-256 checksum How to use checksums |
2f374f0bfd969cc427deabbe4d3f37d5ceb7db5ef84d35a14700707cc8a9f2e6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.5
|
Security for py-json2toon 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.