proto2json 1.0.0

Decode protobuf binary data to JSON without schema definition

proto2json

Decode protobuf binary data to JSON without requiring a schema definition (.proto file).

Features

• 🚀 No Schema Required: Decode protobuf data without .proto files
• 🔍 Type Detection: Automatically detects strings, nested messages, and numeric types
• 📊 Multiple Interpretations: Provides multiple type interpretations for ambiguous fields
• 🎯 Simple API: Easy-to-use interface with just two main functions
• 📦 Zero Dependencies: No external dependencies required

Installation

pip install proto2json

Quick Start

from proto2json import decode_protobuf, decode_protobuf_to_json

# Decode protobuf bytes to Python dict
protobuf_bytes = b'\x08\x96\x01\x12\x07\x74\x65\x73\x74\x69\x6e\x67'
result = decode_protobuf(protobuf_bytes)
print(result)

# Decode to JSON string
json_str = decode_protobuf_to_json(protobuf_bytes)
print(json_str)

Output Format

The decoder returns a dictionary where keys are field numbers (as strings) and values contain:

Wire Type 0 (Varint)

{
    "wire_type": 0,
    "value": 150,
    "as_signed": -150  # Only present if value could be negative
}

Wire Type 1 (Fixed64)

{
    "wire_type": 1,
    "as_fixed64": 123456789,
    "as_sfixed64": -123456789,
    "as_double": 1.23456789
}

Wire Type 2 (Length-Delimited)

String:

{
    "wire_type": 2,
    "type": "string",
    "value": "hello world"
}

Nested Message:

{
    "wire_type": 2,
    "type": "message",
    "value": {
        "1": {"wire_type": 0, "value": 42}
    }
}

Bytes:

{
    "wire_type": 2,
    "type": "bytes",
    "value": "48656c6c6f"  # hex encoded
}

Wire Type 5 (Fixed32)

{
    "wire_type": 5,
    "as_fixed32": 12345,
    "as_sfixed32": -12345,
    "as_float": 123.45
}

API Reference

decode_protobuf(data: bytes) -> Dict[str, Any]

Decode protobuf binary data into a JSON-compatible dictionary.

Parameters:

• data (bytes): Protobuf serialized bytes

Returns:

• Dict with field numbers as string keys and decoded values

Example:

data = b'\x08\x96\x01'  # field 1, varint 150
result = decode_protobuf(data)
# {'1': {'wire_type': 0, 'value': 150}}

decode_protobuf_to_json(data: bytes, indent: int = 2) -> str

Decode protobuf binary data and return as JSON string.

Parameters:

• data (bytes): Protobuf serialized bytes
• indent (int, optional): JSON indentation level. Use None for compact output. Default: 2

Returns:

• JSON string representation of the decoded protobuf

Example:

json_str = decode_protobuf_to_json(data, indent=2)
print(json_str)

Wire Type Constants

The package exports wire type constants for reference:

from proto2json import (
    WIRE_TYPE_VARINT,           # 0
    WIRE_TYPE_FIXED64,          # 1
    WIRE_TYPE_LENGTH_DELIMITED, # 2
    WIRE_TYPE_FIXED32,          # 5
)

Advanced Usage

Handling Repeated Fields

Repeated fields are automatically detected and merged into arrays:

# Protobuf with repeated field
data = b'\x08\x01\x08\x02\x08\x03'  # field 1 appears 3 times
result = decode_protobuf(data)
# {
#   '1': [
#     {'wire_type': 0, 'value': 1},
#     {'wire_type': 0, 'value': 2},
#     {'wire_type': 0, 'value': 3}
#   ]
# }

Nested Messages

Nested messages are recursively decoded:

# Protobuf with nested message
result = decode_protobuf(data)
# {
#   '1': {
#     'wire_type': 2,
#     'type': 'message',
#     'value': {
#       '1': {'wire_type': 0, 'value': 42}
#     }
#   }
# }

How It Works

The decoder uses the protobuf wire format specification to parse the binary data:

1. Field Detection: Reads field numbers and wire types
2. Type Inference:
   • For length-delimited fields, tries to detect if it's a message, string, or raw bytes
   • For fixed-width fields, provides multiple numeric interpretations
3. Recursive Decoding: Nested messages are automatically detected and decoded

Limitations

• Type Ambiguity: Without a schema, some types are ambiguous (e.g., a varint could be int32, int64, bool, or enum)
• Packed Fields: Packed repeated fields are decoded as raw bytes
• Groups: Deprecated protobuf groups are not supported
• Enums: Enum values are decoded as integers

Use Cases

• Debugging: Inspect protobuf messages without access to .proto files
• Reverse Engineering: Analyze unknown protobuf data structures
• Data Recovery: Recover data from protobuf files when schemas are lost
• Testing: Quick protobuf data inspection during development

Examples

See the examples directory for more usage examples.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

Inspired by protobuf_inspector project.

Changelog

v1.0.0 (2026-01-09)

• Initial release
• Support for all standard protobuf wire types
• Automatic type detection for length-delimited fields
• Recursive nested message decoding