Convert from protobuf to arrow and back
Project description
Protarrow
Protarrow is a python library for converting from protobuf to arrow and back.
It is used at Tradewell Technologies, to share data between transactional and analytical applications, with little boilerplate code and zero data loss.
Installation
pip install protarrow
Usage
Taking a simple protobuf:
message MyProto {
string name = 1;
int32 id = 2;
repeated int32 values = 3;
}
It can be converted to a pyarrow.Table
:
import protarrow
my_protos = [
MyProto(name="foo", id=1, values=[1, 2, 4]),
MyProto(name="bar", id=2, values=[3, 4, 5]),
]
table = protarrow.messages_to_table(my_protos, MyProto)
name | id | values |
---|---|---|
foo | 1 | [1 2 4] |
bar | 2 | [3 4 5] |
And the table can be converted back to proto:
protos_from_table = protarrow.table_to_messages(table, MyProto)
See the documentation
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
protarrow-0.1.5.tar.gz
(16.1 kB
view hashes)
Built Distribution
protarrow-0.1.5-py3-none-any.whl
(17.9 kB
view hashes)
Close
Hashes for protarrow-0.1.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b23d0a0bbeb90e116282b4b367d843bd18d260ade793febcad21e90637c57b1c |
|
MD5 | 7e4cf72bb919cb42bb59aedbb98b8452 |
|
BLAKE2b-256 | 0b4db1d5d844fa94f5878ca87049b5b68adf0e47f07aea746b48ad85a9224781 |