Skip to main content

Binary data handling in Python using dictionaries

Project description

pycstruct

AppVeyor Coverage Status Documentation

pycstruct is a python library for converting binary data to and from ordinary python dictionaries or specific instance objects.

Data is defined similar to what is done in C language structs, unions, bitfields and enums.

Typical usage of this library is read/write binary files or binary data transmitted over a network.

Following complex C types are supported:

  • Structs
  • Unions
  • Bitfields
  • Enums

These types may consist of any traditional data types (integer, unsigned integer, boolean and float) between 1 to 8 bytes large, arrays (lists), and strings (ASCII/UTF-8).

Structs, unions, bitfields and enums can be embedded inside other structs/unions in any level.

Individual elements can be stored / read in any byte order and alignment.

pycstruct also supports parsing of existing C language source code to automatically generate the pycstruct definitions / instances.

Checkout the full documentation here.

Installation

Simply install the package using pip:

python3 -m pip install pycstruct

Example

Following C has a structure (person) with a set of elements that are written to a binary file.

#include <stdio.h>
#include <stdbool.h>
#include <string.h>

#pragma pack(1) // To secure no padding is added in struct

struct person 
{ 
    char name[50];
    unsigned int age;
    float height;
    bool is_male;
    unsigned int nbr_of_children;
    unsigned int child_ages[10];
};


void main(void) {
    struct person p;
    memset(&p, 0, sizeof(struct person));

    strcpy(p.name, "Foo Bar");
    p.age = 42;
    p.height = 1.75; // m
    p.is_male = true;
    p.nbr_of_children = 2;
    p.child_ages[0] = 7;
    p.child_ages[1] = 9;

    FILE *f = fopen("simple_example.dat", "w");
    fwrite(&p, sizeof(struct person), 1, f);
    fclose(f);
}

To read the binary file using pycstruct following code required.

import pycstruct

person = pycstruct.StructDef()
person.add('utf-8', 'name', length=50)
person.add('uint32', 'age')
person.add('float32','height')
person.add('bool8', 'is_male')
person.add('uint32', 'nbr_of_children')
person.add('uint32', 'child_ages', length=10)

with open('simple_example.dat', 'rb') as f:
    inbytes = f.read()

# Dictionary representation
result = person.deserialize(inbytes)
print('Dictionary object:')
print(str(result))

# Alternative, Instance representation
instance = person.instance(inbytes)
print('\nInstance object:')
print(f'name: {instance.name}')
print(f'nbr_of_children: {instance.nbr_of_children}')
print(f'child_ages[1]: {instance.child_ages[1]}')

The produced output will be::

{'name': 'Foo Bar', 'is_male': True, 'nbr_of_children': 2, 
 'age': 42, 'child_ages': [7, 9, 0, 0, 0, 0, 0, 0, 0, 0], 
 'height': 1.75}

Instance object:
name: Foo Bar
nbr_of_children: 2
child_ages[1]: 9

To write a binary file from python using the same structure using pycstruct following code is required.

import pycstruct

person = pycstruct.StructDef()
person.add('utf-8', 'name', length=50)
person.add('uint32', 'age')
person.add('float32','height')
person.add('bool8', 'is_male')
person.add('uint32', 'nbr_of_children')
person.add('uint32', 'child_ages', length=10)

# Dictionary representation
mrGreen = {}
mrGreen['name'] = "MR Green"
mrGreen['age'] = 50
mrGreen['height'] = 1.93
mrGreen['is_male'] = True
mrGreen['nbr_of_children'] = 3
mrGreen['child_ages'] = [13,24,12]
buffer = person.serialize(mrGreen)

# Alternative, Instance representation
mrGreen = person.instance()
mrGreen.name = "MR Green"
mrGreen.age = 50
mrGreen.height = 1.93
mrGreen.is_male = True
mrGreen.nbr_of_children = 3
mrGreen.child_ages[0] = 13
mrGreen.child_ages[1] = 24
mrGreen.child_ages[2] = 12
buffer = bytes(mrGreen)

# Write to file
f = open('simple_example_mr_green.dat','wb')
f.write(buffer)
f.close()

Parsing source files

pycstruct also supports parsing C source code defined in external files or defined in strings.

Assume the C code listed in the first example is named simple_example.c. Then you could parse the source code instead of manually creating the definitions:

import pycstruct

definitions = pycstruct.parse_file('simple_example.c')

with open('simple_example.dat', 'rb') as f:
    inbytes = f.read()

# Dictionary representation
result = definitions['person'].deserialize(inbytes)
print(str(result))

# Alternative, Instance representation
instance = definitions['person'].instance(inbytes)

The produced output will be the same as in the first example (the dictionary).

Full documentation

Checkout the full documentation here.

Author and license

This application is written by Joel Midstjärna and is licensed under the MIT License.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pycstruct-0.12.2.tar.gz (35.1 kB view details)

Uploaded Source

Built Distribution

pycstruct-0.12.2-py3-none-any.whl (23.5 kB view details)

Uploaded Python 3

File details

Details for the file pycstruct-0.12.2.tar.gz.

File metadata

  • Download URL: pycstruct-0.12.2.tar.gz
  • Upload date:
  • Size: 35.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.8.10

File hashes

Hashes for pycstruct-0.12.2.tar.gz
Algorithm Hash digest
SHA256 273fc23892b9855d41fcafafa511c5f7d69c7dc54eb683bba5b161ea38305114
MD5 5989e9295872eea89483875df7f34e1b
BLAKE2b-256 b3792ef54f0c3c0fef35fcf1e963436ff88a7706e40a578a8b60dfdef4ba4ae5

See more details on using hashes here.

File details

Details for the file pycstruct-0.12.2-py3-none-any.whl.

File metadata

  • Download URL: pycstruct-0.12.2-py3-none-any.whl
  • Upload date:
  • Size: 23.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.8.10

File hashes

Hashes for pycstruct-0.12.2-py3-none-any.whl
Algorithm Hash digest
SHA256 ddde9816981f1b74cdf3dfa2732eaf2872644f9d7d3cf8794bcf152394274970
MD5 a9371cf64141a7da2634c2172be8558f
BLAKE2b-256 ea4b1834449e4c9eb44e3ed3671d51e79d7e2f04b3f8e87dcff7cf4f8f9c3943

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page