spanned-toml 2.0.1.1

A lil' TOML parser, but with span

Spanned-Toml

A lil' TOML parser, but with span

This project is an extension of @hukkin's Tomli libray, but with span.

A span is simply a Python slice that helps to retrieve where a given object was parsed from.

Motivation

TOML has become a popular format for configuration files, and many tools now rely on parsing such files. However, most parsers error on invalid TOML syntax, not configuration specific errors. E.g., what happens with the following file?

age = -45  # age should be a positive integer

First, you parse the TOML file, which is valid, then you invalidate the age value because it is negative. But how to pinpoint the location of where age was defined to the user?

There is where Spanned-Toml comes into play. For every key / value, you can obtain the span information about where it was define. The span is simple a Python slice, that can be used to index the original TOML string.

Table of Contents generated with mdformat-toc

• Motivation
• Intro
• Installation
• Usage
• Why choosing Spanned-Toml over others?
• Versions

Intro

Spanned-Toml is a Python library for parsing TOML, with the only addition of span information for every object (both keys and values). It is fully compatible with TOML v1.0.0, and its goal is to provide span information with minimal overhead over Tomli.

As such, Spanned-Toml provides the same features and API as Tomli, with the only difference is that it returns a Spanned[dict], instead of dict.

If you whish to get the same output as with Tomli, you can always call .unspan() on a Spanned object.

Installation

pip install spanned-toml

Usage

Toml-Spanned has the exact same interface as Tomli. Therefore, I recommend you checking Tomli's usage.

The only addition is that, instead of returning an object T, it returns Spanned[T], and nested objects are also Spanned, i.e., array and dictionnary values are also spanned.

From Spanned[T], you can always obtain the inner value T with .inner():

import spanned_toml as toml

toml_dict = toml.loads("age = 10")

assert toml_dict["age"].inner() == 10

NOTE: for convenience, Spanned[T] inherits most attributes from T.

If you have nested Spanned objects, then you can call .unspan() to remove all span information, and obtain the same object as if you used Tomli.

toml_str = """
[[players]]
name = "Lehtinen"
number = 26

[[players]]
name = "Numminen"
number = 27
"""

toml_dict = toml.loads(toml_str).unspan()
assert toml_dict == {
    "players": [{"name": "Lehtinen", "number": 26}, {"name": "Numminen", "number": 27}]
}

Last, but not least, you can retrieve the exact part of the string that was used to parse a given key or value.

player_span = toml_dict["players"][0]["name"].span()

assert toml_str[player_span] == '"Lehtinen"'  # Quotes are included in span

NOTE: arrays of tables have an empty span, since then can be defined in multiple parts of a given file.

Why choosing Spanned-Toml over others?

Spanned-Toml was mainly built for another project I am working on.

You should use this package whenever you care about where specific parts in a TOML config file are coming from. This might be useful, e.g., if you want to have a validation layer, on top of the default TOML, and that you want to exactly pinpoint where an error originated.

Otherwise, if you just care about parsing TOML file or speed, then directly use Tomli (or other faster alternatives).

Versions

Toml-Spanned follows the same versions as Tomli, and tries to be in sync with it.

Therefore, Tomli-Spanned's version X.Y.Z.P matches Tomli's version X.Y.Z. The P number of for patches, and is only intended to fix issues related to span.