Fast python conversion from protobuf to arrow using rust
Project description
ptars
Protobuf to Arrow, using Rust
Example
Take a protobuf:
message SearchRequest {
string query = 1;
int32 page_number = 2;
int32 result_per_page = 3;
}
And convert serialized messages directly to pyarrow.RecordBatch
:
messages = [
SearchRequest(
query="protobuf to arrow",
page_number=0,
result_per_page=10,
),
SearchRequest(
query="protobuf to arrow",
page_number=1,
result_per_page=10,
),
]
payloads = [message.SerializeToString() for message in messages]
pool = HandlerPool()
handler = pool.get_for_message(SearchRequest.DESCRIPTOR)
record_batch = handler.list_to_record_batch(payloads)
query | page_number | result_per_page |
---|---|---|
protobuf to arrow | 0 | 10 |
protobuf to arrow | 1 | 10 |
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ptars-0.0.1rc1.tar.gz
(78.4 kB
view hashes)
Built Distributions
Close
Hashes for ptars-0.0.1rc1-cp311-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d5c4ac53f745a2ccd7b96f087ff3dd5db113b2d2bdb30c3adb2fc511d8bab57a |
|
MD5 | 87eee3c238d7a4644aed5c4b05171479 |
|
BLAKE2b-256 | 9670a022b3b21a70e1becce4ad34d68207e2a69efdeac7637fc755e0659ed9c4 |
Close
Hashes for ptars-0.0.1rc1-cp311-none-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 297dbcb2afb3d103dec3ffe961fe47064722cb8999e2bbdc6cba31ba542ef2f1 |
|
MD5 | cc98710ba9e716f9ef5597844f1ab9e8 |
|
BLAKE2b-256 | 0bfc15f7cc661091b6e93fe6b214fc1f61b82697e4b7cc2bb8bc734bac166f60 |
Close
Hashes for ptars-0.0.1rc1-cp311-cp311-macosx_10_12_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 50e2c588cc76e1707bb74f11b86b35da8fa8d6253e448a2ab78d4dde89bd3cec |
|
MD5 | d8d9180a5a00cdec5fb601c77f8a7d0e |
|
BLAKE2b-256 | e8a3155b2c19e49e050a5e7e9122a54f337399b68619b41e5d86e474f040c528 |
Close
Hashes for ptars-0.0.1rc1-cp311-cp311-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5f26a9a5c4fd324ea94188cc02555f76ebca585c0ebcfb080be93282e8a18e45 |
|
MD5 | 52696df295a817f02b54ed6a815313cf |
|
BLAKE2b-256 | b8319d293cf69838b7aebeae566dae02a051d1c73b1039ba0040b28f58c008ed |