pycstructdataparser-lib 1.0.1

Parse C/C++ struct definitions from header text and serialize/deserialize binary data with ctypes — no compilation needed

pycstructdataparser-lib

[中文说明书] | English

Automatically parse struct definitions directly from C/C++ header text and process binary data in Python — No compilation required, no manual ctypes.Structure subclass writing needed, zero boilerplate code. Seamless data transfer between C/C++ and Python with automated data unpacking/packing.

Overview

Specifically, this package uses Python's built-in ctypes module to parse struct definitions from C/C++ .h files and handle their corresponding binary data. You only need to provide the struct definition text from the header file to accomplish binary data serialization (pack) and deserialization (unpack).

With this module, users no longer need to rewrite Python CRUD code each time data field definitions change — packing/unpacking methods can be generated directly from .h files.

Ideal for data communication between Python and C/C++ programs, such as:

• TCP/UDP bitstream message transmission
• Local binary data file read/write (.bin files, .dat files, etc.)
• Project scenarios where data protocols change frequently
• Development stages with rapid protocol iteration

Quick Start

Install directly from PyPI:

pip install pycstructdataparser-lib

Then it's ready to use. The following code demonstrates the complete core workflow:

Define struct → Parse → Pack → Unpack

from pycstructdataparser_lib import CStructParser, pack, unpack

# 1. Prepare C struct definition text
header = '''
#pragma pack(push, 1)
typedef struct {
    unsigned char id;
    unsigned int  value;
    float         data[4];
} MyStruct;
#pragma pack(pop)
'''

# 2. Parse the header file and register the struct type
parser = CStructParser()
parser.parse(header)

# 3. Pack a Python dictionary into binary bytes (serialization)
original = {'id': 10, 'value': 12345, 'data': [1.0, 2.0, 3.0, 4.0]}
raw = pack(parser, 'MyStruct', original)
print(f"pack output: {len(raw)} bytes")

# 4. Unpack binary bytes back into a Python dictionary (deserialization)
restored = unpack(parser, 'MyStruct', raw)
print(restored)

Expected output:

pack output: 21 bytes
{'id': 10, 'value': 12345, 'data': [1.0, 2.0, 3.0, 4.0]}

Features

• Bitfield Support: Parses bitfield declarations in the form unsigned char field : 2;
• C Header Parsing: Recognizes typedef struct { … } TypeName; blocks, automatically extracting struct definitions
• #pragma pack Support: Correctly handles #pragma pack(push, N) and #pragma pack(pop) alignment stack
• Array Support: Handles fixed-size arrays, e.g., Type array[10];
• Nested Structs: Supports struct types used as fields within other structs
• Serialization (pack): Converts Python dictionaries into C struct binary bytes
• Deserialization (unpack): Parses raw bytes back into nested Python dictionaries
• Strict Mode: Optionally validates that all non-bitfield fields are present during packing

Installation Methods

Method 1: PyPI Installation (Recommended)

pip install pycstructdataparser-lib

Method 2: Local Use from GitHub Repository

After cloning the repository, you can install it in editable mode or add the repository directory to the Python search path.

Editable Mode Installation (Recommended):

git clone https://github.com/NGC13009/pycstructdataparser_lib.git
cd pycstructdataparser_lib
pip install -e .

Adding the Path Directly in Code (no installation required):

import sys
sys.path.insert(0, '/path/to/pycstructdataparser_lib')
from pycstructdataparser_lib import CStructParser, pack, unpack

This repository uses a flat module layout — the root directory (containing __init__.py) is itself the package directory. Placing this directory in the Python package search path (sys.path) allows normal imports via from pycstructdataparser_lib import .... Known Issue⚠: Due to this flat structure, the source code may require minor adjustments before use, such as renaming the folder, removing the pycstructdataparser_lib.py file, and migrating its content into __init__.py.

Method 3: Install via .whl Package

Download the latest .whl file from the Release page, then run:

pip install pycstructdataparser_lib-*.whl

Method 4: Manual Build

pip install build  # Build toolchain
git clone https://github.com/NGC13009/pycstructdataparser_lib.git
cd pycstructdataparser_lib
python -m build
pip install dist/pycstructdataparser_lib-*.whl

Usage Guide

Core API

Function / Method	Description
CStructParser()	Creates a parser instance
parser.parse(header_content)	Parses a C/C++ header string, registering the struct types defined within
parser.get_struct_class(name)	Retrieves the corresponding ctypes.Structure subclass by type name
parser.struct_classes	Returns a dictionary of all registered struct classes (property)
pack(parser, struct_name, data_dict, strict=True)	Serializes a Python dictionary into a binary byte sequence
unpack(parser, struct_name, data)	Deserializes a binary byte sequence into a Python dictionary

Basic Usage: Directly Processing Header Strings and Binary Streams

The core workflow of this library is: pass the struct definition text from a C header file to CStructParser.parse(), then use pack() and unpack() to convert between Python dictionaries and binary bytes. See the "Quick Start" example above for details.

Reading/Writing Local Binary Files

Using test_roundtrip.py as a reference, a typical workflow proceeds as follows:

1. Read a header file → Parse it with CStructParser
2. Construct a Python dictionary as test data
3. Call pack() to serialize the dictionary into binary bytes, then write to a .bin file
4. Read bytes from the .bin file, call unpack() to restore them into a dictionary
5. Compare the original data with the restored data to verify consistency

For detailed implementation, refer to the test_roundtrip.py file in this repository. This file also demonstrates operations such as saving a dictionary as a JSON file and packing after loading from JSON.

Application in Network Communication

In TCP/UDP programming, the output of pack() can be sent directly to a socket, and bytes received from a socket can be passed directly to unpack(). Since the library's inputs and outputs are all of bytes type, it is naturally compatible with standard socket interfaces. Business-specific logic (such as handling message boundaries and byte order negotiation) must be implemented by the user according to the actual protocol.

API Reference

CStructParser

class CStructParser:
    def __init__(self) -> None: ...
    def parse(self, header_content: str) -> None: ...
    def get_struct_class(self, name: str) -> type: ...
    @property
    def struct_classes(self) -> Dict[str, type]: ...

parse(header_content)

Parses a C/C++ header string, extracting all typedef struct { … } Name; blocks and dynamically creating a ctypes.Structure subclass for each struct.

Supported features:

• C-style comments (// and /* … */)
• #pragma pack(push, N) / #pragma pack(pop) alignment directives
• Standard C basic types, bitfields, fixed-size arrays
• Nested named structs (forward references are handled preferentially)

get_struct_class(name)

Returns the ctypes.Structure subclass corresponding to the specified name. Raises ValueError if the name is not registered.

struct_classes

Returns a dictionary of all registered struct classes, with struct names as keys and corresponding ctypes.Structure subclasses as values.

pack(parser, struct_name, data_dict, strict=True)

def pack(
    parser: CStructParser,
    struct_name: str,
    data_dict: Dict[str, Any],
    strict: bool = True,
) -> bytes: ...

Serializes a Python dictionary into binary bytes corresponding to a C struct.

• parser: A CStructParser instance on which parse() has been called.
• struct_name: The target struct type name.
• data_dict: A dictionary mapping field names to field values.
• strict: If True (default), raises ValueError when non-bitfield fields are missing; if False, missing fields are silently filled with 0.

unpack(parser, struct_name, data)

def unpack(
    parser: CStructParser,
    struct_name: str,
    data: bytes,
) -> Dict[str, Any]: ...

Deserializes raw binary bytes into a nested Python dictionary.

• parser: A CStructParser instance on which parse() has been called.
• struct_name: The target struct type name.
• data: The binary byte sequence to decode.

Supported C Type Mapping

C Type	ctypes Type
char, signed char	c_byte
unsigned char	c_ubyte
short, short int	c_short
unsigned short	c_ushort
int, signed int	c_int
unsigned int	c_uint
long, long int	c_long
unsigned long	c_ulong
long long, long long int	c_longlong
unsigned long long	c_ulonglong
float	c_float
double	c_double
long double	c_longdouble
int8_t .. uint64_t	c_int8 .. c_uint64
float32_t	c_float
float64_t	c_double

Comparison with Common Approaches

Feature	struct module	ctypes.Structure	construct library	cStructDataParser
Direct parsing from header text	❌	❌	❌	✅
Automatic #pragma pack alignment	❌	❌	❌	✅
Bitfield support	❌	✅	✅	✅
Nested structs	❌	Manual writing required	✅	✅
No external dependencies (pure Python)	✅	✅	✅	✅

Known Limitations

• Currently only supports little-endian byte order; big-endian scenarios are not yet handled
• If the device uses big-endian order, additional byte order processing is needed before/after calling pack() / unpack()

License

This project is licensed under GPLv3.0.