jtd-to-proto 0.3.0

A tool to dynamically create protobuf message classes from JSON Typedef

JTD To Proto

This library holds utilities for converting JSON Typedef to Protobuf.

Why?

The protobuf langauge is a powerful tool for defining language-agnostic, composable datastructures. JSON Typedef (JTD) is also a powerful tool to accomplish the same task. Both have advantages and disadvantages that make each fit better for certain use cases. For example:

• Protobuf:
  • Advantages
    • Compact serialization
    • Auto-generated grpc client and service libraries
    • Client libraries can be used from different programming languages
  • Disadvantages
    • Learning curve to understand the full ecosystem
    • Not a familiar tool outside of service engineering
    • Static compilation step required to use in code
• JTD:
  • Advantages
    • Can be learned in 5 minutes
    • Can be written inline in the programming language of choice (e.g. as a dict in python)
  • Disadvantages
    • No optimized serialization beyond json
    • No automated service implementations
    • Static jtd-codegen step needed to generate native structures

This project aims to bring them together so that a given project can take advantage of the best of both:

• Define your structures in JTD for simplicity
• Dynamically create google.protobuf.Descriptor objects to allow for protobuf serialization and deserialization
• Reverse render a .proto file from the generated Descriptor so that stubs can be generated in other languages
• No static compiliation needed!

Usage

The usage of this library can be best understood with a simple example:

import jtd_to_proto

# Declare the Foo protobuf message class
Foo = jtd_to_proto.descriptor_to_message_class(
    jtd_to_proto.jtd_to_proto(
        name="Foo",
        package="foobar",
        jtd_def={
            "properties": {
                # Bool field
                "foo": {
                    "type": "boolean",
                },
                # Array of nested enum values
                "bar": {
                    "elements": {
                        "enum": ["EXAM", "JOKE_SETTING"],
                    }
                }
            }
        },
    )
)

# Declare an object that references Foo as the type for a field
Bar = jtd_to_proto.descriptor_to_message_class(
    jtd_to_proto.jtd_to_proto(
        name="Bar",
        package="foobar",
        jtd_def={
            "properties": {
                "baz": {
                    "type": Foo.DESCRIPTOR,
                },
            },
        },
    ),
)

def write_protos(proto_dir: str):
    """Write out the .proto files for Foo and Bar to the given directory"""
    Foo.write_proto_file(proto_dir)
    Bar.write_proto_file(proto_dir)

Similar Projects

There are a number of similar projects in this space that offer slightly diferent value:

• jtd-codegen: This project focuses on statically generating language-native code (including python) to represent the JTD schema.
• py-json-to-proto: This project aims to deduce a schema from an instance of a json object.
• pure-protobuf: This project has a very similar aim to jtd-to-proto, but it skips the intermediate descriptor representation and thus is not able to produce native message.Message classes.