Skip to main content

json2toon

Convert JSON structures into TOON (Token-Oriented Object Notation) and back again, with tooling to measure token savings and create ready-to-send prompts for LLMs.

At a glance

  • JSON <-> TOON conversion (json_to_toon, toon_to_json, convert_file)
  • Configurable formatting (indent, separators, quoting, table layouts)
  • Automatic table layout for uniform object arrays
  • Token counting and comparison reports
  • Prompt helpers to wrap TOON payloads for LLMs
  • Rich CLI with syntax highlighting and reports
  • Fully covered by pytest (39 passing tests)

Install

pip install json2toon

CLI usage

The CLI is exposed as json2toon.

# JSON -> TOON
json2toon to-toon input.json -o output.toon

# TOON -> JSON (pretty by default)
json2toon to-json input.toon -o output.json

# Token comparison report
json2toon report input.json -f table
json2toon report input.json -f json   # machine-readable

Flags of interest:

  • -c/--config load a JSON config file (see Config section)
  • -p/--pretty pretty-print TOON or JSON when writing to stdout
  • -P/--no-pretty disable pretty JSON output
  • -o/--output write to a file instead of stdout

Examples:

# Convert and view with highlighting
json2toon to-toon sample.json --pretty

# Round trip via files
json2toon to-toon sample.json -o sample.toon
json2toon to-json sample.toon -o sample.roundtrip.json

# Get a JSON report of token savings
json2toon report sample.json -f json > stats.json

Python API

Import from json2toon:

from json2toon import (
    json_to_toon, toon_to_json, convert_file, get_conversion_stats,
    ToonEncoder, ToonDecoder, ToonConfig,
    compare_formats, count_tokens, generate_report,
    create_llm_prompt, create_response_template, wrap_in_code_fence, add_system_prompt,
)

Core helpers (core.py)

  • json_to_toon(data, config=None) -> str: Encode any JSON-serializable Python data to TOON.
  • toon_to_json(toon_str, config=None) -> Any: Decode TOON back to Python data.
  • convert_file(input_path, output_path, to_toon=True, config=None): File-level conversion in either direction.
  • get_conversion_stats(data, config=None, output_format="text") -> dict: Compute token counts, savings, and a formatted report (text/json/markdown).

Encoder (encoder.py)

  • ToonEncoder.encode(data) -> str: Main entry. Handles primitives, objects, arrays, tables, indentation, string quoting/escaping, and inline vs block arrays.
  • Table layout: for uniform arrays of dicts (based on uniformity_threshold and min_table_rows), arrays render as ASCII tables.

Decoder (decoder.py)

  • ToonDecoder.decode(toon_str) -> Any: Parses tables, list items, objects, primitives, inline JSON, and escaped strings/newlines.

Config (config.py)

  • ToonConfig fields:
    • separator (default :)
    • table_separator (default |), header_separator
    • max_inline_array_length, compress_primitive_arrays
    • max_string_length, quote_strings
    • indent_size, max_nesting_depth
    • uniformity_threshold, min_table_rows
  • get_default_config() returns defaults.
  • save_config(config, path), load_config(path) to persist/restore.

Analysis (analyzer.py)

  • analyze_structure(data, config) -> StructureInfo: Classifies primitives, objects, arrays, and detects uniform arrays.
  • is_uniform_array(arr, threshold=0.8) -> (bool, keys) used by the encoder.
  • should_use_table_format(data, config) -> bool selects table layout.

Metrics (metrics.py)

  • count_tokens(text, encoding_name="cl100k_base") -> int: Uses tiktoken to count tokens.
  • compare_formats(data, encoder, encoding_name="cl100k_base") -> ComparisonResult: Token counts for JSON vs TOON.
  • generate_report(comparison, output_format="text") -> str: Formats text/json/markdown reports.

Prompt helpers (prompt.py)

  • create_llm_prompt(toon_str, system_prompt=None): Wrap TOON payloads for model input.
  • create_response_template(): Skeleton for expected model replies.
  • wrap_in_code_fence(text, language="toon"): Add triple-backtick fences.
  • add_system_prompt(prompt, system_prompt): Prepend system guidance.

CLI (cli.py)

  • Commands: to-toon, to-json, report. All support optional config loading and pretty-print toggles. Success output uses ASCII-only markers for Windows compatibility.

Exceptions (exceptions.py)

EncodingError, DecodingError, AnalysisError, ConfigurationError all derive from Json2ToonError.

Usage examples

Programmatic round trip

from json2toon import json_to_toon, toon_to_json

data = {"name": "Alice", "scores": [95, 87, 92]}
toon = json_to_toon(data)
restored = toon_to_json(toon)
assert restored == data

Custom config

from json2toon import ToonConfig, ToonEncoder

config = ToonConfig(indent_size=4, quote_strings=True, table_separator='|')
encoder = ToonEncoder(config)
toon = encoder.encode({"users": [{"id": 1, "name": "Bob"}, {"id": 2, "name": "Eve"}]})
print(toon)

Token savings report

from json2toon import ToonEncoder, compare_formats, generate_report

data = {"items": [
    {"id": 1, "name": "Alpha"},
    {"id": 2, "name": "Beta"},
]}
encoder = ToonEncoder()
comparison = compare_formats(data, encoder)
print(generate_report(comparison, output_format="markdown"))

Build an LLM prompt

from json2toon import json_to_toon, create_llm_prompt

data = {"request": "Summarize", "payload": {"text": "Hello world"}}
toon = json_to_toon(data)
prompt = create_llm_prompt(toon, system_prompt="You are a concise assistant.")
print(prompt)

Configuration file example

Save as toon_config.json:

{
  "separator": ":",
  "table_separator": "|",
  "header_separator": "-",
  "max_inline_array_length": 8,
  "compress_primitive_arrays": true,
  "quote_strings": true,
  "indent_size": 2,
  "uniformity_threshold": 0.75,
  "min_table_rows": 2
}

Use it:

json2toon to-toon data.json -c toon_config.json -o data.toon

Project layout

src/json2toon/
  analyzer.py     # structure detection, uniformity checks
  cli.py          # Typer-based CLI
  config.py       # ToonConfig dataclass + load/save
  core.py         # top-level helpers and file conversion
  decoder.py      # TOON -> Python
  encoder.py      # Python -> TOON
  exceptions.py   # custom exception types
  metrics.py      # token counting and reports
  prompt.py       # LLM prompt utilities
tests/            # pytest suite (39 tests)

Development

python -m pytest           # run tests
python -m pytest --cov     # run with coverage

The suite exercises all public APIs (encode/decode, configs, CLI, metrics, prompt helpers) and passes on Windows with ASCII-only CLI output.

Metadata

Release files for py-json2toon 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for py-json2toon 0.1.0
File Size Uploaded Provenance Actions
py_json2toon-0.1.0.tar.gz 21.4 kB Without provenance Details for py_json2toon-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for py-json2toon 0.1.0
File Interpreter ABI Platform Actions
py_json2toon-0.1.0-py3-none-any.whl Python 3 none any Details for py_json2toon-0.1.0-py3-none-any.whl

Total release size: 38.7 kB

Release files / py_json2toon-0.1.0.tar.gz

Download URL py_json2toon-0.1.0.tar.gz
Size 21.4 kB
Tags Source
SHA-256 checksum
How to use checksums
0497b367bc2b902c82ab99665b90cd066b504f876cb7e2fd71ab18afd6553f5b
BLAKE2b-256 checksum
How to use checksums
506c7835e97aa943efd2f605744fb84217353141bae12754a47c39583f671bef
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.5

Release files / py_json2toon-0.1.0-py3-none-any.whl

Download URL py_json2toon-0.1.0-py3-none-any.whl
Size 17.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b3ea1319bac0fa051564f4d92ef883d6e1d93c4c66062b3e3eddbd7cec7acde2
BLAKE2b-256 checksum
How to use checksums
2f374f0bfd969cc427deabbe4d3f37d5ceb7db5ef84d35a14700707cc8a9f2e6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.5

Security for py-json2toon 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page