jsno 1.0.7

Convert Python data to and from json-compatible data structures

jsno

Convert Python data to and from JSON-compatible data structures.

Jsno provides functions that turn any Python values into JSON-compatible structures and back. The jsonified data can then be used wherever JSON data is required: it can be dumped into a file, sent over the network in an API call, or stored in a database that support JSON data.

Note that jsno does not replace the standard json module - it does not produce JSON encoded string, but instead turns arbitrary Python values into structures that can be JSON encoded.

Jsno provides jsonification for most of the standard Python datatypes. It is also easily extensible, so that custom jsonification can be defined for new datatypes.

Jsno has special support for dataclasses, so no effort is needed to make jsonificatiin work for classes defined as dataclasses.

Basic usage

Data is converted to JSON strutures using the jsonify function:

from datetime import datetime, date
import jsno

jsonified = jsno.jsonify({"date": datetime.utcnow().date()})
assert jsonified == {"date": "2023-07-30"}

Converting the jsonified data back is done using the unjsonify function. However, unjsonify can't guess the type of the resulting value that the JSON data should be converted to. So it needs to be given it explicitly, using "bracket syntax":

value = jsno.unjsonify[dict[str, date]](jsonified)
assert value == {"date": datetime.date(2023, 7, 30)}

Here the example value had to be typed as a dictionary with date values. When working with heterogenous data, it's usually best to define it as dataclasses instead.

Example with dataclasses

First, let's define a custom dataclass to keep track of domain data:

from dataclasses import dataclass

@dataclass
class DomainRecord:
    domain: str
    ips: list[str]
    enabled_at: date | None = None

Then, let's make a couple of such records:

domains = [
    DomainRecord(
        domain="example.com",
        ips=["93.184.216.34"],
        enabled_at=date(1992, 1, 1)
    ),
    DomainRecord(
        domain="another.example.com",
        ips=[]
    )
]

With jsno, these can be turned into JSON and stored in a local file:

import json
import pathlib

jsonified = jsno.jsonify(domains)
pathlib.Path('domains.json').write_text(json.dumps(jsonified, indent=4))

The file will look like this:

[
    {
        "domain": "example.com",
        "ips": ["93.184.216.34"],
        "enabled_at": "1992-01-01"
    },
    {
        "domain": "another.example.com",
        "ips": []
    }
]

To read and use the domains later:

jsonified = json.loads(pathlib.Path('domains.json').read_text())
domains = jsno.unjsonify[list[DomainRecord]](jsonified)

assert domains[0].enabled_at.year == 1992

Supported types

Native JSON types

The following types are native JSON types, and they are mapped to themselves when jsonifying and unjsonifying:

• str
• bool
• int
• float
• NoneType (None maps to JSON's null)
• list (JSON array)
• dict (JSON object)

Abstract base classes

The operations are also defined to the following abstract base classes, so any type implementing them can be jsnoified and unjsonified:

• Sequence (maps to list)
• Mapping (maps to dict)
• Set (maps to list)
• ByteString (maps to base64 encoded string)

Union types

Union types (typing.Union and T | U) are supported. However, when unjsonising union types, it's important that the JSON representations of two different types do not overlap. Otherwise, jsno will choose to unjsonify ambiguous data to the first type in the union that matches.

Dataclasses

Dataclasses are supported. They get converted to JSON objects. The name of the class is not included in the result, unless the dataclass is a member of a variant family.

Other standard Python types

• tuples
• ranges
• enums
• date and datetime objects (converted to ISO-formatted strings)
• complex
• decimal.Decimal (converted to JSON strings)
• pathlib.Path
• zoneinfo.ZoneInfo
• Literal (only int and str literals)

Dumps and loads

Jsno provides shortcut functions dumps and loads with interface that is similar to the standard json module function.s jsno.dumps both jsonifies its argument and turns it into a JSON-encoded string, similar to standard json.dumps function. Correspondinly, jsno.loads both decodes and unjsonify JSON data.

json = jsno.dumps(date(2023, 7, 30))
assert json == "2023-07-30"

day = jsno.loads[date](json)
assert day == date(2023, 7, 30)

Defining custom jsonification

Jsno's jsonify and unjsonify are defined as singledispatch functions, so implementations to new types can be registered using a decorator.

For example, Python's standard random number generator can be given json representation:

from random import Random


@jsno.jsonify.register(Random)
def _(value):
    # jsonify the random number generator's state
    state = value.getstate()
    return jsno.jsonify(state)


@jsno.unjsonify.register(Random)
def _(value, as_type):
    # first unjsonify the state
    state = jsno.unjsonify[tuple[int, tuple[int, ...], float | None]](value)

    # create a new Random object and install the unjsonified state
    result = Random()
    result.setstate(state)
    return result

Now it's possible to for example have a random number generator as a part of a data structure defined using dataclasses, and have it converted to and from JSON automatically:

@dataclass
class GameState:
    identifier: str
    board: list[tuple[int, int, str]]
    rng: Random


def save_game(database, state: GameState):
    database.store(state.identifier, jsonify(state))


def load_game(database, identifier: str) -> GameState:
    json = database.load(identifier)
    return unjsonify[GameState](json)

Constraints

Jsno support annotating types with constraints that are boolean valued functions that must return True for the unjsonified value for it to be valid. For example, to only accept email addresses that contain the "@" character:

from typing import Annotated
from jsno import Constraint

@dataclass
class User:
    username: str
    email: Annotated[str, Constraint(lambda it: "@" in it)]

The most typical constraints are those that limit the value or the length of a property to a certain range. For these, jsno provides predefined shortcuts:

@dataclass
class Player:
    username: Annotated[str, Constraint.len(min=4, max=16)]
    credits: Annotated[int, Constraint.range(min=0)]

Variant families

Sometimes it's useful to have a hierarchy of classes, consisting of several subclasses that need to be stored in JSON. A typical example is defining the AST (abstract syntax tree) for a domain-specific language. Here's the AST of a simple expression language, defined using a hierarchy of dataclasses:

class Expression:
    pass


@dataclass
class LiteralInt(Expression):
    value: int

    def evaluate(self, context):
        return self.value


@dataclass
class BinaryOperator(Expression):
    left: Expression
    right: Expression


class Add(BinaryOperator):
    def evaluate(self, context):
        return self.left.evaluate(context) + self.right.evaluate(context)


class Multiply(BinaryOperator):
    def evaluate(self, context):
        return self.left.evaluate(context) * self.right.evaluate(context)

Now, it's possible to create syntax trees, evaluate them, and convert them into JSON:

ast = Add(LiteralInt(1), Multiply(LiteralInt(2), Reference("x")))

assert ast.evaluate({"x": 3}) == 7

json = jsno.dumps(ast, indent=4)

json will contain:

{
    "left": {
        "value": 1
    },
    "right": {
        "left": {
            "value": 2
        },
        "right": {
            "name": "x"
        }
    }
}

However, reading the AST back from JSON won't work. Trying to unjsonify it as an Expression fails:

jsno.unjsonify[Expression](json)
# TypeError: Unjsonify not defined for <class 'Expression'>

Even if you defined the base Expression class as a dataclass, jsno wouldn't not which of the subclasses should it choose.

The solution to the problem is to mark the root of the expression as the root of a variant family, using the variantfamily decorator:

@jsno.variantfamily(label="type")
class Expression:
    pass

When a class is marked to form a variant family, jsno will add a label to the data jsonfied from it's, or it's subclasses' instances, to identify the class.

Adter adding the decorator, the JSON produced by jsonify contains labels to differentiate between the subclasses (variants):

{
    "type": "Add",
    "left": {
        "type": "LiteralInt",
        "value": 1
    },
    "right": {
        "type": "Multiply",
        "left": {
            "type": "LiteralInt",
            "value": 2
        },
        "right": {
            "type": "Reference",
            "name": "x"
        }
    }
}

The variant label's name (type) was given in the decorator.

Now, jsno is able to unjsonify the AST:

unjsonified_ast = jsno.unjsonify[Expression](json)
assert unjsonified_ast == ast

By default, the label for a class is taken from the class name. It's also possible to give it explicitly using the variantlabel decorator:

@jsno.variantlabel('mult')
class Multiply(Expression):
    # ...

Installation

Install jsno with pip:

pip install jsno

Jsno has no 3rd party dependencies.