blackboxprotobuf_py 0.1.1

A lightweight, zero-dependency Python library to decode, modify, and re-encode arbitrary Protocol Buffers without needing the original `.proto` schema definitions. It is ideal for security research, reverse engineering API payloads, and automated blackbox testing of protobuf endpoints.

blackboxprotobuf_py

A lightweight, zero-dependency Python library to decode, modify, and re-encode arbitrary Protocol Buffers without needing the original .proto schema definitions. It is ideal for security research, reverse engineering API payloads, and automated blackbox testing of protobuf endpoints.

The entire library is written in a single, high-performance, and easily maintainable Python file, making it highly portable and simple to integrate.

---

Features

• Schema Inferences & Decoding: Instantly parse protobuf binaries into Python dictionaries alongside a dynamic schema definition (typedef map).
• Modification & Encoding: Modify python data structures and rebuild valid protobuf binaries using their updated typedef mapping.
• JSON Round-Trip: Export protobuf data to readable JSON and rebuild it back to exact protobuf binary structure.
• Proto Schema Export & Import:
  • Export dynamically inferred schemas directly to a standard .proto file.
  • Import external .proto files to bootstrap schemas and use defined types.
• Payload Transport Wrappers: Built-in, automated detection and codec support for:
  • GZIP compressed message buffers.
  • gRPC HTTP/2 transport packet frames (including multi-packet streams).

---

Installation

pip install blackboxprotobuf_py

---

Quickstart & Examples

1. Basic Protobuf Decoding and Encoding

You can decode any raw protobuf byte stream. If no schema is provided, the library will infer it dynamically:

import blackboxprotobuf_py

# Raw protobuf binary data
protobuf_bytes = b'\x08*\x12\x0bhello world'

# Decode without a predefined schema
data, typedef = blackboxprotobuf_py.decode_message(protobuf_bytes)

print("Decoded Data:")
print(data)
# Output: {'1': 42, '2': 'hello world'}

print("\nInferred Typedef:")
print(typedef)
# Output: {'1': {'type': 'int'}, '2': {'type': 'string'}}

# Modify the decoded python dictionary
data['1'] = 99
data['2'] = 'protobuf simplified'

# Encode it back using the typedef map
new_bytes = blackboxprotobuf_py.encode_message(data, typedef)

---

2. JSON Serialization with Inferred Typedefs

Convert binary protobufs to JSON for human readability or manipulation in web tooling, and parse them back securely:

import blackboxprotobuf_py

protobuf_bytes = b'\x08*\x12\x0bhello world'

# Convert binary to JSON
json_str, inferred_typedef = blackboxprotobuf_py.protobuf_to_json(protobuf_bytes)
print(json_str)
# Output:
# {
#   "1": 42,
#   "2": "hello world"
# }

# Parse modified JSON back to protobuf binary using the inferred typedef
modified_json = '{\n  "1": 100,\n  "2": "json modifications"\n}'
modified_bytes = blackboxprotobuf_py.protobuf_from_json(modified_json, inferred_typedef)

---

3. Exporting and Importing .proto Schema Files

Exporting to .proto

import blackboxprotobuf_py

# Define or infer a typedef mapping
typedef = {
    "1": {"type": "int", "name": "user_id"},
    "2": {"type": "string", "name": "username"}
}
typedef_map = {"UserProfile": typedef}

# Export to proto string
proto_content = blackboxprotobuf_py.export_proto(typedef_map, pkg="my.app")
print(proto_content)
# Output:
# syntax = "proto3";
# package my.app;
#
# message UserProfile {
#   int64 user_id = 1;
#   string username = 2;
# }

Importing from .proto

import blackboxprotobuf_py

proto_schema = """
syntax = "proto3";
package my.app;

message UserProfile {
  int64 user_id = 1;
  string username = 2;
}
"""

# Import schema and register the typedef mapping
imported_typedefs = blackboxprotobuf_py.import_proto(blackboxprotobuf_py.default_config, input_string=proto_schema)
user_profile_typedef = imported_typedefs["my.app.UserProfile"]

---

4. Handling gRPC & GZIP Framed Payloads

blackboxprotobuf_py provides utility functions to decode and encode framed payloads directly from the root module:

import blackboxprotobuf_py

# Automatic decoder detection
payload_bytes = b'\x1f\x8b\x08...'  # Gzip headers
decoders = blackboxprotobuf_py.find_decoders(payload_bytes)

# Decompress Gzip payloads
decompressed_data, encoding_type = blackboxprotobuf_py.decode_payload(payload_bytes, "gzip")

# Encode back to Gzip
compressed_bytes = blackboxprotobuf_py.encode_payload(decompressed_data, "gzip")

For gRPC HTTP/2 framing:

import blackboxprotobuf_py

# Decode gRPC framed message stream
messages, frame_type = blackboxprotobuf_py.decode_payload(grpc_bytes, "grpc")

# Re-encode message stream back with gRPC headers
grpc_framed_bytes = blackboxprotobuf_py.encode_payload(messages, "grpc")

---

License

This project is open-source software. Refer to the codebase headers and LICENSE for details.