pycstruct 0.8.1

Binary data handling in Python using dictionaries

pycstruct

pycstruct is a python library for converting binary data to and from ordinary python dictionaries or specific instance objects.

Data is defined similar to what is done in C language structs, unions, bitfields and enums.

Typical usage of this library is read/write binary files or binary data transmitted over a network.

Following complex C types are supported:

• Structs
• Unions
• Bitfields
• Enums

These types may consist of any traditional data types (integer, unsigned integer, boolean and float) between 1 to 8 bytes large, arrays (lists), and strings (ASCII/UTF-8).

Structs, unions, bitfields and enums can be embedded inside other structs/unions in any level.

Individual elements can be stored / read in any byte order and alignment.

pycstruct also supports parsing of existing C language source code to automatically generate the pycstruct definitions / instances.

Checkout the full documentation here.

Installation

Simply install the package using pip:

python3 -m pip install pycstruct

Example

Following C has a structure (person) with a set of elements that are written to a binary file.

#include <stdio.h>
#include <stdbool.h>
#include <string.h>

#pragma pack(1) // To secure no padding is added in struct

struct person 
{ 
    char name[50];
    unsigned int age;
    float height;
    bool is_male;
    unsigned int nbr_of_children;
    unsigned int child_ages[10];
};


void main(void) {
    struct person p;
    memset(&p, 0, sizeof(struct person));

    strcpy(p.name, "Foo Bar");
    p.age = 42;
    p.height = 1.75; // m
    p.is_male = true;
    p.nbr_of_children = 2;
    p.child_ages[0] = 7;
    p.child_ages[1] = 9;

    FILE *f = fopen("simple_example.dat", "w");
    fwrite(&p, sizeof(struct person), 1, f);
    fclose(f);
}

To read the binary file using pycstruct following code required.

import pycstruct

person = pycstruct.StructDef()
person.add('utf-8', 'name', length=50)
person.add('uint32', 'age')
person.add('float32','height')
person.add('bool8', 'is_male')
person.add('uint32', 'nbr_of_children')
person.add('uint32', 'child_ages', length=10)

with open('simple_example.dat', 'rb') as f:
    inbytes = f.read()

# Dictionary representation
result = person.deserialize(inbytes)
print('Dictionary object:')
print(str(result))

# Alternative, Instance representation
instance = person.instance(inbytes)
print('\nInstance object:')
print('name: {}'.format(instance.name))
print('nbr_of_children: {}'.format(instance.nbr_of_children))
print('child_ages[1]: {}'.format(instance.child_ages[1]))

The produced output will be::

{'name': 'Foo Bar', 'is_male': True, 'nbr_of_children': 2, 
 'age': 42, 'child_ages': [7, 9, 0, 0, 0, 0, 0, 0, 0, 0], 
 'height': 1.75}

Instance object:
name: Foo Bar
nbr_of_children: 2
child_ages[1]: 9

To write a binary file from python using the same structure using pycstruct following code is required.

import pycstruct

person = pycstruct.StructDef()
person.add('utf-8', 'name', length=50)
person.add('uint32', 'age')
person.add('float32','height')
person.add('bool8', 'is_male')
person.add('uint32', 'nbr_of_children')
person.add('uint32', 'child_ages', length=10)

# Dictionary representation
mrGreen = {}
mrGreen['name'] = "MR Green"
mrGreen['age'] = 50
mrGreen['height'] = 1.93
mrGreen['is_male'] = True
mrGreen['nbr_of_children'] = 3
mrGreen['child_ages'] = [13,24,12]
buffer = person.serialize(mrGreen)

# Alternative, Instance representation
mrGreen = person.instance()
mrGreen.name = "MR Green"
mrGreen.age = 50
mrGreen.height = 1.93
mrGreen.is_male = True
mrGreen.nbr_of_children = 3
mrGreen.child_ages[0] = 13
mrGreen.child_ages[1] = 24
mrGreen.child_ages[2] = 12
buffer = bytes(mrGreen)

# Write to file
f = open('simple_example_mr_green.dat','wb')
f.write(buffer)
f.close()

Parsing source files

pycstruct also supports parsing C source code defined in external files or defined in strings.

Assume the C code listed in the first example is named simple_example.c. Then you could parse the source code instead of manually creating the definitions:

import pycstruct

definitions = pycstruct.parse_file('simple_example.c')

with open('simple_example.dat', 'rb') as f:
    inbytes = f.read()

# Dictionary representation
result = definitions['person'].deserialize(inbytes)
print(str(result))

# Alternative, Instance representation
instance = definitions['person'].instance(inbytes)

The produced output will be the same as in the first example (the dictionary).

Full documentation

Checkout the full documentation here.

Author and license

This application is written by Joel Midstjärna and is licensed under the MIT License.