Skip to main content

enhancement for the python package 'construct' that adds support for dataclasses.

Project description

construct-dataclasses

python Codestyle License PyPI

This small repository is an enhancement of the python package construct, which is a powerful tool to declare symmetrical parsers and builders for binary data. This project combines construct with python's dataclasses with support for nested structs.

Installation

You can install the package via pip or just copy the python file (__init__.py) as it is only one.

pip install construct-dataclasses

Usage

More usage examples are placed in the examples/ directory.

Define dataclasses

Before we can start declaring fields on a dataclass, the class itself has to be created. Currently, there are two ways on how to create a dataclass usable by this package.

  1. Use the standard @dataclass decorator and create the parser instance afterwards (recommended for type checking):

    from construct_dataclasses import DataclassStruct
    
    @dataclasses.dataclass
    class Foo: ...
    
    # Create the parser manually
    parser = DataclassStruct(Foo)
    instance = parser.parse(...)
    
  2. Use the @dataclass_struct decorator to define a new dataclass and automatically create a parser instance that will be assigned as a class attribute:

    from construct_dataclasses import dataclass_struct
    
    @dataclass_struct
    class Foo: ...
    
    # Use the class-parser to parse
    instance = Foo.parser.parse(...)
    # or to build
    data = Foo.parser.build(instance)
    

Hint: Use @container to mimic a construct container instance if needed. That may be the case if you have to access an already parsed object of a custom type:

@container
@dataclasses.dataclass
class ImageHeader:
  length: int = csfield(Int32ub)

@dataclasses.dataclass
class Image:
  header: ImageHeader = csfield(ImageHeader)
  data: bytes = csfield(Bytes(this.header.length))

The access to header.length would throw an exception without the container annotation.

Define fields

This module defines a new way how to declare fields of a dataclass. In order to combine the python package construct with python's dataclasses module, this project introduces the following four methods:

  • csfield: Default definition of a field using a subcon or other dataclass

    @dataclass_struct
    class ImageHeader:
        signature: bytes = csfield(cs.Const(b"BMP"))
        num_entries: int = csfield(cs.Int32ul)
    
  • subcsfield: Definition of nested constructs that are contained in list-like structures.

    @dataclass_struct
    class Image:
        header: ImageHeader = csfield(ImageHeader) # dataclass reference
        width: int          = csfield(cs.Int8ub)
        height: int         = csfield(cs.Int8ub)
        # Note that we have to convert our dataclass into a struct using
        # the method "to_struct(...)"
        pixels: list[Pixel] = subcsfield(Pixel, cs.Array(this.width * this.height, to_struct(Pixel)))
    
  • tfield: a simple typed field that tries to return an instance of the given model class. Use subcsfield for dataclass models, csenumfor simple enum fields and tfield for enum types in list fields.

    @dataclass_struct
    class ImageHeader:
        orientations: list[Orientation] = tfield(Orientation, cs.Enum(cs.Int8ul, Orientation))
    
  • csenum: shortcut for simple enum fields

    @dataclass_struct
    class ImageHeader:
        orientations: Orientation = csenum(Orientation, cs.Int8ul)
    

Convert dataclasses

By default, all conversion is done automatically if you don't use instances of SubContruct classes in your field definitions. If you have to define a subcon that needs a nested subcon, like Array or RepeatUntil and you would like to parse a dataclass struct, it is required to convert the defined dataclass into a struct.

  • to_struct: This method converts all fields defined in a dataclass into a single Struct or AlignedStruct instance.

    @dataclass_struct
    class Pixel:
        data: int = csfield(cs.Int8ub)
    
    pixel_struct: construct.Struct = to_struct(Pixel)
    
  • to_object: In order to use data returned by Struct.parse, this method can be used to apply this data and create a dataclass object from it.

    data = pixel_struct.parse(b"...")
    pixel = to_object(data, Pixel)
    

The complete example is shown below:

# Example modifed from here: https://github.com/timrid/construct-typing/
import dataclasses
import enum
import construct as cs

from construct_dataclasses import dataclass_struct, csfield, to_struct, subcsfield, csenum

class Orientation(enum.IntEnum):
    NONE = 0
    HORIZONTAL = 1
    VERTICAL = 2

@dataclass_struct
class ImageHeader:
    signature: bytes         = csfield(cs.Const(b"BMP"))
    orientation: Orientation = csenum(Orientation, cs.Int8ub)

@dataclass_struct
class Pixel:
    data: int = csfield(cs.Int8ub)

@dataclass_struct
class Image:
    header: ImageHeader = csfield(ImageHeader)
    width: int          = csfield(cs.Int8ub)
    height: int         = csfield(cs.Int8ub)
    pixels: list[Pixel] = subcsfield(Pixel, cs.Array(this.width * this.height, to_struct(Pixel)))

obj = Image(
    header=ImageHeader(
        orientation=Orientation.VERTICAL
    ),
    width=3,
    height=2,
    pixels=[Pixel(1), Pixel(2), Pixel(3), Pixel(4), Pixel(5), Pixel(6)]
)

print(Image.parser.build(obj))
print(Image.parser.parse(b"BMP\x02\x03\x02\x01\x02\x03\x04\x05\06"))

The expected output would be:

b'BMP\x02\x03\x02\x01\x02\x03\x04\x05\x06'
Image(
    header=ImageHeader(signature=b'BMP', orientation=<Orientation.VERTICAL: 2>),
    width=3, height=2,
    pixels=[Pixel(data=1), Pixel(data=2), Pixel(data=3), Pixel(data=4), Pixel(data=5), Pixel(data=6)]
)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

construct-dataclasses-1.1.10.tar.gz (23.0 kB view hashes)

Uploaded Source

Built Distribution

construct_dataclasses-1.1.10-py3-none-any.whl (22.3 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page