Protobuf Decoder
Simple protobuf decoder for python
Motivation
The goal of this project is decode protobuf binary without proto files
Installation
Install using pip
pip install protobuf-decoder
Simple Examples
"""
# proto
message Test1 {
string a = 1;
}
# message
{
"a": "테스트"
}
# binary
0A 09 ED 85 8C EC 8A A4 ED 8A B8
"""
from protobuf_decoder.protobuf_decoder import Parser
test_target = "0A 09 ED 85 8C EC 8A A4 ED 8A B8"
parsed_data = Parser().parse(test_target)
assert parsed_data == ParsedResults([ParsedResult(field=1, wire_type="string", data='테스트')])
assert parsed_data.to_dict() == {'results': [{'field': 1, 'wire_type': 'string', 'data': '테스트'}]}
"""
# proto
message Test1 {
int32 a = 1;
}
message Test2 {
Test1 b = 3;
}
# message
{
"a": {
"b": 150
}
}
# binary
1a 03 08 96 01
"""
from protobuf_decoder.protobuf_decoder import Parser
test_target = "1a 03 08 96 01"
parsed_data = Parser().parse(test_target)
assert parsed_data == ParsedResults(
[
ParsedResult(field=3, wire_type="length_delimited", data=ParsedResults([
ParsedResult(field=1, wire_type="varint", data=150)
]))
])
assert parsed_data.to_dict() == {'results': [{'field': 3, 'wire_type': 'length_delimited', 'data': {
'results': [{'field': 1, 'wire_type': 'varint', 'data': 150}]}}]}
"""
# proto
message Test1 {
required string a = 1;
}
# message
{
"a": "✊"
}
# binary
0A 03 E2 9C 8A
"""
from protobuf_decoder.protobuf_decoder import Parser
test_target = "0A 03 E2 9C 8A"
parsed_data = Parser().parse(test_target)
assert parsed_data == ParsedResults([ParsedResult(field=1, wire_type="string", data='✊')])
assert parsed_data.to_dict() == {'results': [{'field': 1, 'wire_type': 'string', 'data': '✊'}]}
Nested Protobuf Detection Logic
Our project implements a distinct method to determine whether a given input is possibly a nested protobuf.
The core of this logic is the is_maybe_nested_protobuf function.
We recently enhanced this function to provide a more accurate distinction and handle nested protobufs effectively.
Current Logic
The is_maybe_nested_protobuf function works by:
- Attempting to convert the given hex string to UTF-8.
- Checking the ordinal values of the first four characters of the converted data.
- Returning
Trueif the data might be a nested protobuf based on certain conditions, otherwise returning False.
Extensibility
You can extend or modify the is_maybe_nested_protobuf function based on your specific requirements or use-cases.
If you find a scenario where the current logic can be further improved,
feel free to adapt the function accordingly.
(A big shoutout to @fuzzyrichie for their significant contributions to this update!)
Remain Bytes
If there are remaining bytes after parsing, the parser will return the remaining bytes as a string.
from protobuf_decoder.protobuf_decoder import Parser
test_target = "ed 85 8c ec 8a a4 ed 8a b8"
parsed_data = Parser().parse(test_target)
assert parsed_data.has_results is False
assert parsed_data.has_remain_data is True
assert parsed_data.remain_data == "ed 85 8c ec 8a a4 ed 8a b8"
assert parsed_data.to_dict() == {'results': [], 'remain_data': 'ed 85 8c ec 8a a4 ed 8a b8', }
Reference
Metadata
Release files for protobuf-decoder 0.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| protobuf_decoder-0.4.0.tar.gz | 7.2 kB | Without provenance | Details for protobuf_decoder-0.4.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| protobuf_decoder-0.4.0-py3-none-any.whl | Python 3 | none | any | Details for protobuf_decoder-0.4.0-py3-none-any.whl |
Total release size: 14.8 kB
Release files / protobuf_decoder-0.4.0.tar.gz
| Download URL | protobuf_decoder-0.4.0.tar.gz |
|---|---|
| Size | 7.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
fa52959cb6feeae61404446218b9eb94f1723a5260d5174d21f715d5c60e74af
|
|
BLAKE2b-256 checksum How to use checksums |
9cbe8da2e35123829cabc41051e493636fa4fcea08a89cb42cd9214147f3546c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/60.2.0 requests-toolbelt/0.9.1 tqdm/4.57.0 CPython/3.8.12
|
Release files / protobuf_decoder-0.4.0-py3-none-any.whl
| Download URL | protobuf_decoder-0.4.0-py3-none-any.whl |
|---|---|
| Size | 7.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
32b7abd4fde7cafe99c9a491a3eeee715a059ba5b6b5d32c67c2918d5bddb174
|
|
BLAKE2b-256 checksum How to use checksums |
e08926381ac04a3e5857e17b6fa6dfc17d371d778f3bd11ff7bfa5dbc032865d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/60.2.0 requests-toolbelt/0.9.1 tqdm/4.57.0 CPython/3.8.12
|
Security for protobuf-decoder 0.4.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.