Skip to main content

Load and dump data from json-like format into typed data structures

Project description

typedload

Load and dump json-like data into typed data structures in Python3, enforcing a schema on the data.

This module provides an API to load dictionaries and lists (usually loaded from json) into Python's NamedTuples, dataclass, sets, enums, and various other typed data structures; respecting all the type-hints and performing type checks or casts when needed.

It can also dump from typed data structures to json-like dictionaries and lists.

It is very useful for projects that use Mypy and deal with untyped data like json, because it guarantees that the data will follow the specified schema.

It is released with a GPLv3 license but it is possible to ask for LGPLv3.

GPLv3 logo

Donate to LtWorf

Example

For example this dictionary, loaded from a json:

data = {
    'users': [
        {
            'username': 'salvo',
            'shell': 'bash',
            'sessions': ['pts/4', 'tty7', 'pts/6']
        },
        {
            'username': 'lop'
        }
    ],
}

Can be treated more easily if loaded into this type:

@dataclasses.dataclass
class User:
    username: str
    shell: str = 'bash'
    sessions: List[str] = dataclasses.field(default_factory=list)

class Logins(NamedTuple):
    users: List[User]

And the data can be loaded into the structure with this:

t_data = typedload.load(data, Logins)

And then converted back:

data = typedload.dump(t_data)

Supported types

Since this is not magic, not all types are supported.

The following things are supported:

  • Basic python types (int, str, bool, float, NoneType)
  • NamedTuple
  • Enum
  • Optional[SomeType]
  • List[SomeType]
  • Dict[TypeA, TypeB]
  • Tuple[TypeA, TypeB, TypeC] and Tuple[SomeType, ...]
  • Set[SomeType]
  • Union[TypeA, TypeB]
  • dataclass
  • attr.s
  • ForwardRef (Refer to the type in its own definition)
  • Literal
  • TypedDict
  • datetime.date, datetime.time, datetime.datetime
  • re.Pattern
  • Path
  • IPv4Address, IPv6Address
  • typing.Any
  • typing.NewType
  • uuid.UUID

Unions

typedload works fine with untagged unions. However using Literal fields to tag them makes it much faster.

Using Mypy

Mypy and similar tools work without requiring any plugins.

# This is treated as Any, no checks done.
data = json.load(f)

# This is treated as Dict[str, int]
# but there will be runtime errors if the data does not
# match the expected format
data = json.load(f)  # type: Dict[str, int]

# This is treated as Dict[str, int] and an exception is
# raised if the actual data is not Dict[str, int]
data = typedload.load(json.load(f), Dict[str, int])

So when using Mypy, it makes sense to make sure that the type is correct, rather than hoping the data will respect the format.

Extending

Type handlers can easily be added, and existing ones can be replaced, so the library is fully cusomizable and can work with any type.

Inheriting a base class is not required.

Install

  • pip install typedload
  • apt install python3-typedload
  • Latest and greatest .deb file is in releases

Documentation

The tests are hard to read but provide more in depth examples of the capabilities of this module.

Used by

As dependency, typedload is used by those entities. Feel free to add to the list.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

typedload-2.29.tar.gz (76.2 kB view details)

Uploaded Source

Built Distribution

typedload-2.29-py3-none-any.whl (53.7 kB view details)

Uploaded Python 3

File details

Details for the file typedload-2.29.tar.gz.

File metadata

  • Download URL: typedload-2.29.tar.gz
  • Upload date:
  • Size: 76.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.11.9

File hashes

Hashes for typedload-2.29.tar.gz
Algorithm Hash digest
SHA256 b24446b9d8a5002caf3a1a30ce6f231ae7a228bf0326ead9eb59f76fd146bab1
MD5 238cf9eb6a8d1d2ec0b703fba5cb9350
BLAKE2b-256 2071734f608d77a6614e491e732affc45e8229131bf293cbd4e394ffa4061630

See more details on using hashes here.

File details

Details for the file typedload-2.29-py3-none-any.whl.

File metadata

  • Download URL: typedload-2.29-py3-none-any.whl
  • Upload date:
  • Size: 53.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.11.9

File hashes

Hashes for typedload-2.29-py3-none-any.whl
Algorithm Hash digest
SHA256 4b2d0a0c88a9f830d4b1aacef62024f3f66d01d09be9bdafb81eb21c23e021a0
MD5 67a0d851e107bb218397fce3d12c6941
BLAKE2b-256 8b2fa12c3d37521c53b6b77ff59eaeb7f618e3a3447c6b6bf7b4e87d60c7c1f9

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page