Skip to main content

TOON for Python

CI PyPI Python versions TOON spec Documentation

The official Python implementation of TOON (Token-Oriented Object Notation): a compact, human-readable encoding of the JSON data model, designed to save tokens in LLM prompts. Keep working with dictionaries and lists in your code, and encode them as TOON only where a model reads them.

  • Complete: implements TOON specification 4.4 and passes every official conformance fixture.
  • Familiar: dumps, dump, loads, and load, like the json module, plus encode and decode, the names of the TypeScript reference implementation.
  • Lossless: loads(dumps(data)) == data for every JSON-compatible value.
  • Pure Python, no dependencies, fully typed, Python 3.10+.

Installation

pip install toon-format
# or
uv add toon-format

The module is toon_format. Install toon-format[pydantic] for the Pydantic integration.

Quick start

import toon_format

data = {
    "context": {"task": "Our favorite hikes together", "location": "Boulder"},
    "friends": ["ana", "luis", "sam"],
    "hikes": [
        {"id": 1, "name": "Blue Lake Trail", "distanceKm": 7.5, "wasSunny": True},
        {"id": 2, "name": "Ridge Overlook", "distanceKm": 9.2, "wasSunny": False},
    ],
}

text = toon_format.dumps(data)
print(text)
context:
  task: Our favorite hikes together
  location: Boulder
friends[3]: ana,luis,sam
hikes[2]{id,name,distanceKm,wasSunny}:
  1,Blue Lake Trail,7.5,true
  2,Ridge Overlook,9.2,false
assert toon_format.loads(text) == data

Keys of uniform arrays are written once, as a header, and each object becomes a row. With OpenAI's o200k_base tokenizer, this document takes 65 tokens, against 76 for compact JSON and 130 for indented JSON. The saving grows with the number of rows: 100 uniform records take 38% fewer tokens than compact JSON. Deeply nested or irregular data gains less.

Usage

Options

Encoding and decoding follow the json module, with the same keyword arguments where they apply:

toon_format.dumps(data, delimiter="\t")  # tabs often tokenize better than commas
toon_format.dumps(data, default=str)  # fallback for unsupported types
toon_format.loads(text, parse_float=Decimal)  # same hooks as json.loads
toon_format.loads(llm_output, strict=False)  # tolerate wrong counts and indentation

Errors

Invalid documents raise toon_format.ToonDecodeError, a ValueError that carries the 1-based line and the source line of the error:

try:
    toon_format.loads("a: 1\ntags[3]: a,b")
except toon_format.ToonDecodeError as exc:
    print(exc.line, exc.source)  # 2 tags[3]: a,b
    print(exc)  # line 2: Declared length 3 but found 2

encode and decode

encode and decode are the names used by the TypeScript reference implementation, @toon-format/toon, and by the other TOON implementations. Here they are aliases of dumps and loads, so code and examples carry over from one language to the other:

from toon_format import decode, encode

data = {
    "users": [
        {"id": 1, "name": "Ada", "role": "admin"},
        {"id": 2, "name": "Bob", "role": "user"},
    ]
}

print(encode(data))
# users[2]{id,name,role}:
#   1,Ada,admin
#   2,Bob,user

assert decode(encode(data)) == data

Options are keyword arguments in snake case, a spelling the specification allows (§13):

TypeScript Python
encode(data) encode(data)
encode(data, { indentSize: 4, delimiter: '\t' }) encode(data, indent_size=4, delimiter="\t")
decode(text) decode(text)
decode(text, { strict: false }) decode(text, strict=False)

The streaming functions (encodeLines, decodeStream) and the replacer option have no Python counterpart.

Pydantic

With Pydantic, toon_format.dumps accepts any model, and ToonPydanticModel adds TOON counterparts of the JSON helpers:

from toon_format.pydantic import ToonPydanticModel


class User(ToonPydanticModel):
    name: str
    age: int


user = User.model_validate_toon("name: Ada\nage: 36")

Command line

This package has no command; use the official @toon-format/cli, which runs with Node.js and needs no installation:

npx @toon-format/cli data.json -o data.toon   # JSON to TOON
npx @toon-format/cli data.toon -o data.json   # TOON to JSON
npx @toon-format/cli data.json --stats        # token savings against JSON

Documentation

The full documentation is on Read the Docs: usage, the Python type mapping, the choices the specification leaves to implementations, and Pydantic.

Migrating from toon_format 0.9

The distribution and the module keep their names, and encode(value) and decode(text) keep working. Options are now keyword arguments instead of a dictionary, the token helpers and the toon command are gone, and the output follows specification 4.4. See the migration guide.

Contributing

See CONTRIBUTING.md. Project conventions for humans and coding agents are in AGENTS.md.

License

MIT

Metadata

Release files for toon-format 1.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for toon-format 1.2.0
File Size Uploaded Provenance Actions
toon_format-1.2.0.tar.gz 20.8 kB Without provenance Details for toon_format-1.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for toon-format 1.2.0
File Interpreter ABI Platform Actions
toon_format-1.2.0-py3-none-any.whl Python 3 none any Details for toon_format-1.2.0-py3-none-any.whl

Total release size: 43.6 kB

Release files / toon_format-1.2.0.tar.gz

Download URL toon_format-1.2.0.tar.gz
Size 20.8 kB
Tags Source
SHA-256 checksum
How to use checksums
ed09c0cb8ba58f17372f876df1e43400d2d916aef67c641f152d8a28c5898669
BLAKE2b-256 checksum
How to use checksums
069e88b94e00ae3589130bb2bde233c60b55776deb670cc3d077657d6a8b9a2e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.1 {"installer":{"name":"uv","version":"0.13.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / toon_format-1.2.0-py3-none-any.whl

Download URL toon_format-1.2.0-py3-none-any.whl
Size 22.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d94db4b088a281a7bbc85e75d7fbd24e6a0ec7bc110edb129285259eca6522c0
BLAKE2b-256 checksum
How to use checksums
f39e69331571d5b91b3f34f0dcd7c8135bb3241e41c3c3effb4af8f57772a9ca
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.1 {"installer":{"name":"uv","version":"0.13.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for toon-format 1.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page