Skip to main content

enhancement for the python package 'construct' that adds support for dataclasses.

Project description

construct-dataclasses

python Codestyle License PyPI

This small repository is an enhancement of the python package construct, which is a powerful tool to declare symmetrical parsers and builders for binary data. This project combines construct with python's dataclasses with support for nested structs.

Installation

You can install the package via pip or just copy the python file (__init__.py) as it is only one.

pip install construct-dataclasses

Usage

More usage examples are placed in the examples/ directory.

Define dataclasses

Before we can start declaring fields on a dataclass, the class itself has to be created. Currently, there are two ways on how to create a dataclass usable by this package.

  1. Use the standard @dataclass decorator and create the parser instance afterwards (recommended for type checking):

    from construct_dataclasses import DataclassStruct
    
    @dataclasses.dataclass
    class Foo: ...
    
    # Create the parser manually
    parser = DataclassStruct(Foo)
    instance = parser.parse(...)
    
  2. Use the @dataclass_struct decorator to define a new dataclass and automatically create a parser instance that will be assigned as a class attribute:

    from construct_dataclasses import dataclass_struct
    
    @dataclass_struct
    class Foo: ...
    
    # Use the class-parser to parse
    instance = Foo.parser.parse(...)
    # or to build
    data = Foo.parser.build(instance)
    

Hint: Use @container to mimic a construct container instance if needed. That may be the case if you have to access an already parsed object of a custom type:

@container
@dataclasses.dataclass
class ImageHeader:
  length: int = csfield(Int32ub)

@dataclasses.dataclass
class Image:
  header: ImageHeader = csfield(ImageHeader)
  data: bytes = csfield(Bytes(this.header.length))

The access to header.length would throw an exception without the container annotation.

Define fields

This module defines a new way how to declare fields of a dataclass. In order to combine the python package construct with python's dataclasses module, this project introduces the following four methods:

  • csfield: Default definition of a field using a subcon or other dataclass

    @dataclass_struct
    class ImageHeader:
        signature: bytes = csfield(cs.Const(b"BMP"))
        num_entries: int = csfield(cs.Int32ul)
    
  • subcsfield: Definition of nested constructs that are contained in list-like structures.

    @dataclass_struct
    class Image:
        header: ImageHeader = csfield(ImageHeader) # dataclass reference
        width: int          = csfield(cs.Int8ub)
        height: int         = csfield(cs.Int8ub)
        # Note that we have to convert our dataclass into a struct using
        # the method "to_struct(...)"
        pixels: list[Pixel] = subcsfield(Pixel, cs.Array(this.width * this.height, to_struct(Pixel)))
    
  • tfield: a simple typed field that tries to return an instance of the given model class. Use subcsfield for dataclass models, csenumfor simple enum fields and tfield for enum types in list fields.

    @dataclass_struct
    class ImageHeader:
        orientations: list[Orientation] = tfield(Orientation, cs.Enum(cs.Int8ul, Orientation))
    
  • csenum: shortcut for simple enum fields

    @dataclass_struct
    class ImageHeader:
        orientations: Orientation = csenum(Orientation, cs.Int8ul)
    

Convert dataclasses

By default, all conversion is done automatically if you don't use instances of SubContruct classes in your field definitions. If you have to define a subcon that needs a nested subcon, like Array or RepeatUntil and you would like to parse a dataclass struct, it is required to convert the defined dataclass into a struct.

  • to_struct: This method converts all fields defined in a dataclass into a single Struct or AlignedStruct instance.

    @dataclass_struct
    class Pixel:
        data: int = csfield(cs.Int8ub)
    
    pixel_struct: construct.Struct = to_struct(Pixel)
    
  • to_object: In order to use data returned by Struct.parse, this method can be used to apply this data and create a dataclass object from it.

    data = pixel_struct.parse(b"...")
    pixel = to_object(data, Pixel)
    

The complete example is shown below:

# Example modifed from here: https://github.com/timrid/construct-typing/
import dataclasses
import enum
import construct as cs

from construct_dataclasses import dataclass_struct, csfield, to_struct, subcsfield, csenum

class Orientation(enum.IntEnum):
    NONE = 0
    HORIZONTAL = 1
    VERTICAL = 2

@dataclass_struct
class ImageHeader:
    signature: bytes         = csfield(cs.Const(b"BMP"))
    orientation: Orientation = csenum(Orientation, cs.Int8ub)

@dataclass_struct
class Pixel:
    data: int = csfield(cs.Int8ub)

@dataclass_struct
class Image:
    header: ImageHeader = csfield(ImageHeader)
    width: int          = csfield(cs.Int8ub)
    height: int         = csfield(cs.Int8ub)
    pixels: list[Pixel] = subcsfield(Pixel, cs.Array(this.width * this.height, to_struct(Pixel)))

obj = Image(
    header=ImageHeader(
        orientation=Orientation.VERTICAL
    ),
    width=3,
    height=2,
    pixels=[Pixel(1), Pixel(2), Pixel(3), Pixel(4), Pixel(5), Pixel(6)]
)

print(Image.parser.build(obj))
print(Image.parser.parse(b"BMP\x02\x03\x02\x01\x02\x03\x04\x05\06"))

The expected output would be:

b'BMP\x02\x03\x02\x01\x02\x03\x04\x05\x06'
Image(
    header=ImageHeader(signature=b'BMP', orientation=<Orientation.VERTICAL: 2>),
    width=3, height=2,
    pixels=[Pixel(data=1), Pixel(data=2), Pixel(data=3), Pixel(data=4), Pixel(data=5), Pixel(data=6)]
)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

construct-dataclasses-1.1.10.tar.gz (23.0 kB view details)

Uploaded Source

Built Distribution

construct_dataclasses-1.1.10-py3-none-any.whl (22.3 kB view details)

Uploaded Python 3

File details

Details for the file construct-dataclasses-1.1.10.tar.gz.

File metadata

File hashes

Hashes for construct-dataclasses-1.1.10.tar.gz
Algorithm Hash digest
SHA256 31070c3bcb25c7fb128b3a7f13592bbc699082b5977518eacf38066aeb352b9f
MD5 d218407d60df5ac7a29dfba85056f5f0
BLAKE2b-256 9577c92e415abb80daf7ddf53fb16638cfdcc68b71446a30db844e8f6783ccaf

See more details on using hashes here.

File details

Details for the file construct_dataclasses-1.1.10-py3-none-any.whl.

File metadata

File hashes

Hashes for construct_dataclasses-1.1.10-py3-none-any.whl
Algorithm Hash digest
SHA256 cf00d1edc015342a299690d71107a4837133e9a66b1dad0082c18b54b86b1d52
MD5 52ceb4f27805824b32f4ec752027d53f
BLAKE2b-256 cca25be00f122a0f00a116a59dc757a19eab099ec7f777ecca5651f5386c4267

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page