Binary data handling in Python using dictionaries
Project description
pycstruct
pycstruct is a python library for converting binary data to and from ordinary python dictionaries or specific instance objects.
Data is defined similar to what is done in C language structs, unions, bitfields and enums.
Typical usage of this library is read/write binary files or binary data transmitted over a network.
Following complex C types are supported:
- Structs
- Unions
- Bitfields
- Enums
These types may consist of any traditional data types (integer, unsigned integer, boolean and float) between 1 to 8 bytes large, arrays (lists), and strings (ASCII/UTF-8).
Structs, unions, bitfields and enums can be embedded inside other structs/unions in any level.
Individual elements can be stored / read in any byte order and alignment.
pycstruct also supports parsing of existing C language source code to automatically generate the pycstruct definitions / instances.
Checkout the full documentation here.
Installation
Simply install the package using pip:
python3 -m pip install pycstruct
Example
Following C has a structure (person) with a set of elements that are written to a binary file.
#include <stdio.h>
#include <stdbool.h>
#include <string.h>
#pragma pack(1) // To secure no padding is added in struct
struct person
{
char name[50];
unsigned int age;
float height;
bool is_male;
unsigned int nbr_of_children;
unsigned int child_ages[10];
};
void main(void) {
struct person p;
memset(&p, 0, sizeof(struct person));
strcpy(p.name, "Foo Bar");
p.age = 42;
p.height = 1.75; // m
p.is_male = true;
p.nbr_of_children = 2;
p.child_ages[0] = 7;
p.child_ages[1] = 9;
FILE *f = fopen("simple_example.dat", "w");
fwrite(&p, sizeof(struct person), 1, f);
fclose(f);
}
To read the binary file using pycstruct following code required.
import pycstruct
person = pycstruct.StructDef()
person.add('utf-8', 'name', length=50)
person.add('uint32', 'age')
person.add('float32','height')
person.add('bool8', 'is_male')
person.add('uint32', 'nbr_of_children')
person.add('uint32', 'child_ages', length=10)
with open('simple_example.dat', 'rb') as f:
inbytes = f.read()
# Dictionary representation
result = person.deserialize(inbytes)
print('Dictionary object:')
print(str(result))
# Alternative, Instance representation
instance = person.instance(inbytes)
print('\nInstance object:')
print(f'name: {instance.name}')
print(f'nbr_of_children: {instance.nbr_of_children}')
print(f'child_ages[1]: {instance.child_ages[1]}')
The produced output will be::
{'name': 'Foo Bar', 'is_male': True, 'nbr_of_children': 2,
'age': 42, 'child_ages': [7, 9, 0, 0, 0, 0, 0, 0, 0, 0],
'height': 1.75}
Instance object:
name: Foo Bar
nbr_of_children: 2
child_ages[1]: 9
To write a binary file from python using the same structure using pycstruct following code is required.
import pycstruct
person = pycstruct.StructDef()
person.add('utf-8', 'name', length=50)
person.add('uint32', 'age')
person.add('float32','height')
person.add('bool8', 'is_male')
person.add('uint32', 'nbr_of_children')
person.add('uint32', 'child_ages', length=10)
# Dictionary representation
mrGreen = {}
mrGreen['name'] = "MR Green"
mrGreen['age'] = 50
mrGreen['height'] = 1.93
mrGreen['is_male'] = True
mrGreen['nbr_of_children'] = 3
mrGreen['child_ages'] = [13,24,12]
buffer = person.serialize(mrGreen)
# Alternative, Instance representation
mrGreen = person.instance()
mrGreen.name = "MR Green"
mrGreen.age = 50
mrGreen.height = 1.93
mrGreen.is_male = True
mrGreen.nbr_of_children = 3
mrGreen.child_ages[0] = 13
mrGreen.child_ages[1] = 24
mrGreen.child_ages[2] = 12
buffer = bytes(mrGreen)
# Write to file
f = open('simple_example_mr_green.dat','wb')
f.write(buffer)
f.close()
Parsing source files
pycstruct also supports parsing C source code defined in external files or defined in strings.
Assume the C code listed in the first example is named simple_example.c. Then you could parse the source code instead of manually creating the definitions:
import pycstruct
definitions = pycstruct.parse_file('simple_example.c')
with open('simple_example.dat', 'rb') as f:
inbytes = f.read()
# Dictionary representation
result = definitions['person'].deserialize(inbytes)
print(str(result))
# Alternative, Instance representation
instance = definitions['person'].instance(inbytes)
The produced output will be the same as in the first example (the dictionary).
Full documentation
Checkout the full documentation here.
Author and license
This application is written by Joel Midstjärna and is licensed under the MIT License.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file pycstruct-0.12.2.tar.gz
.
File metadata
- Download URL: pycstruct-0.12.2.tar.gz
- Upload date:
- Size: 35.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.8.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 273fc23892b9855d41fcafafa511c5f7d69c7dc54eb683bba5b161ea38305114 |
|
MD5 | 5989e9295872eea89483875df7f34e1b |
|
BLAKE2b-256 | b3792ef54f0c3c0fef35fcf1e963436ff88a7706e40a578a8b60dfdef4ba4ae5 |
File details
Details for the file pycstruct-0.12.2-py3-none-any.whl
.
File metadata
- Download URL: pycstruct-0.12.2-py3-none-any.whl
- Upload date:
- Size: 23.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.8.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ddde9816981f1b74cdf3dfa2732eaf2872644f9d7d3cf8794bcf152394274970 |
|
MD5 | a9371cf64141a7da2634c2172be8558f |
|
BLAKE2b-256 | ea4b1834449e4c9eb44e3ed3671d51e79d7e2f04b3f8e87dcff7cf4f8f9c3943 |