Skip to main content

NumProto provides NumPy ndarray to protobuf conversion.

Project description

NumProto

NumProto is a simple python3.6+ library to serialize and deserialize numpy arrays into and from protobuf messages.

Installation

You can install NumProto from the PyPI package:

$ pip install numproto

Usage

NumProto serializes a numpy array into an NDArray message as specified in ndarray.proto:

syntax = "proto3";

package numproto.protobuf;

message NDArray {
    bytes ndarray = 1;
}

This library provides two methods: one for serialization ndarray_to_proto and one for deserialization proto_to_ndarray.

import numpy as np

from numproto import ndarray_to_proto, proto_to_ndarray

nda = np.arange(10)

serialized_nda = ndarray_to_proto(nda)
deserialized_nda = proto_to_ndarray(serialized_nda)

assert np.array_equal(nda, deserialized_nda)

Re-using the proto files in another project

Re-distributing *.proto files is notoriously difficult. In order to make this easier the ndarray.proto file is installed together with the python package.

This allows us to simply pass the site-packages path as an import path to protoc.

To get the correct path of site-packages check the Location key when running pip show numproto.

For example let's say we want to create a new proto file and import NDArray to use within one of our defined messages:

syntax = "proto3";

import "numproto/protobuf/ndarray.proto";

message MyMessage {
    numproto.protobuf.NDArray my_array = 1;
}

And to compile using grpcio-tools simply do:

$ python -m grpc_tools.protoc -I/usr/lib/python3.6/site-packages/ -I./ --python_out=. --grpc_python_out=. my_proto.proto

(you may need to adjust the location of site-packages)

Tests

To run the tests first install the numproto package from source in development mode and then run the tests using pytest:

$ git clone https://github.com/xainag/numproto.git
$ cd numproto
$ pip install -e .[dev]

$ pytest

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for numproto, version 0.3.0
Filename, size File type Python version Upload date Hashes
Filename, size numproto-0.3.0-py3-none-any.whl (9.7 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size numproto-0.3.0.tar.gz (4.6 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page