Usage

1. Convert the proto file to python Schematics classes:

   protobuf_schematics <path-to-file.proto> generated_schematics_proto.py # or any output filename

2. Convert your ProtoBuf message to Json:

In Java:

import com.google.protobuf.util.JsonFormat;

FileWriter file = new FileWriter("protoBufMessage.json")
JsonFormat.Printer printer = JsonFormat.printer().preservingProtoFieldNames();
String message = printer.print(someProtoBufMessage);
file.write(message)

or from Python:

import json
from google.protobuf.json_format import MessageToJson

json = MessageToJson(org, preserving_proto_field_name=True, including_default_value_fields=True)
with open("protoBufMessage.json", 'w') as output:
    json.dump(json, output)

3. In your project, load the message in python as Schematics object:

import json
from generated_schematics_proto import SomeMessage # import the schematics message class

schematics_root_message = SomeMessage(json.load(open('protoBufMessage.json')))

Or use a message loaded in python by Protobuf:

import json
from generated_schematics_proto import SomeMessage # import the schematics message class

# ... get your protobuf message as the pb class representation
schematics_root_message = SomeMessage.import_from_protobuf_message(protobuf_message)

Example

This proto file:

syntax = "proto3";

enum IPAddressFamily {
    INVALID = 0;
    IPv4 = 1;
    IPv6 = 2;
};

message ProtocolAndPorts {
    repeated uint32 ports = 3;
}

message FlowFilter {
    enum SomeEnum {
        VALUE = 0;
    };
    string id = 1 [deprecated = true];
    SomeEnum consumer_filter_id = 2;
    map<string, ProtocolAndPorts> ports = 3;
    repeated ProtocolAndPorts protocol_and_ports = 4;
}

Will be converted to:

class IPAddressFamily(Enum):
    INVALID = 0
    IPv4 = 1
    IPv6 = 2


class ProtocolAndPorts(ProtobufMessageModel):
    ports = ListType(IntType())


class FlowFilter(ProtobufMessageModel):
    class InnerEnum(Enum):
        VALUE = 0

    id = StringType()
    consumer_filter_id = EnumType(SomeEnum)
    ports = DictType(ModelType(ProtocolAndPorts), str)
    protocol_and_ports = ListType(ModelType(ProtocolAndPorts))

Features

• Support both Protobuf syntax 2 and 3.

• Support builtin types such as StringType, IntType.

• Support proto map fields as Schematics DictType.

• Support repeated modifier as convert to ListType.

• Support Enum class generation and custom Schematics EnumType.

• Support custom schematics ByteArrayType base64 encoded byte arrays converted from Java.

Development

First, install the Pipfile and create the proper virtual environment:

pipenv install --dev

To check linting with flake8, run:

make lint

To run the unittests against your working python version:

py.test

To see coverage report:

make coverage

To run tests against all supported python versions:

tox

To make the docs (which will be automatically published to readthedocs on commits to the master branch):

make docs

Credits

The parsing work of .proto files is provided thanks to the awesome guys at PyroBuf.

This package was created with Cookiecutter and the elgertam/cookiecutter-pipenv project template, based on audreyr/cookiecutter-pypackage.

TODO

• Add more meaningful unittests.

• Improve Protobuf support and integration.

• add support for parsing the Protobuf generated files as it is more comprehensive than the current Pyrobuf support.

Read for Release

0.4.0 (2019-01-16)

• Added more meaningful unittests.

• Export Types and Models out of the templates.

• Export heavy definition logic (such as field definition) from the jinja template to pythonic filters.

• Improve integration with Protobuf.

0.2.0 (2019-01-14)

• Improve README - Add usage examples.

• Update the setup.py to point to official PyroBuf 0.8.5 PyPi release.

0.1.0 (2019-01-13)

• First release on PyPI.

• CLI interface for compiling a given proto file to schematics definition file.