Skip to main content

A tool to dynamically create protobuf message classes from JSON Typedef

Project description

JTD To Proto

This library holds utilities for converting JSON Typedef to Protobuf.

Why?

The protobuf langauge is a powerful tool for defining language-agnostic, composable datastructures. JSON Typedef (JTD) is also a powerful tool to accomplish the same task. Both have advantages and disadvantages that make each fit better for certain use cases. For example:

  • Protobuf:
    • Advantages
      • Compact serialization
      • Auto-generated grpc client and service libraries
      • Client libraries can be used from different programming languages
    • Disadvantages
      • Learning curve to understand the full ecosystem
      • Not a familiar tool outside of service engineering
      • Static compilation step required to use in code
  • JTD:
    • Advantages
      • Can be learned in 5 minutes
      • Can be written inline in the programming language of choice (e.g. as a dict in python)
    • Disadvantages
      • No optimized serialization beyond json
      • No automated service implementations
      • Static jtd-codegen step needed to generate native structures

This project aims to bring them together so that a given project can take advantage of the best of both:

  • Define your structures in JTD for simplicity
  • Dynamically create google.protobuf.Descriptor objects to allow for protobuf serialization and deserialization
  • Reverse render a .proto file from the generated Descriptor so that stubs can be generated in other languages
  • No static compiliation needed!

Usage

The usage of this library can be best understood with a simple example:

import jtd_to_proto

# Declare the Foo protobuf message class
Foo = jtd_to_proto.descriptor_to_message_class(
    jtd_to_proto.jtd_to_proto(
        name="Foo",
        package="foobar",
        jtd_def={
            "properties": {
                # Bool field
                "foo": {
                    "type": "boolean",
                },
                # Array of nested enum values
                "bar": {
                    "elements": {
                        "enum": ["EXAM", "JOKE_SETTING"],
                    }
                }
            }
        },
    )
)

# Declare an object that references Foo as the type for a field
Bar = jtd_to_proto.descriptor_to_message_class(
    jtd_to_proto.jtd_to_proto(
        name="Bar",
        package="foobar",
        jtd_def={
            "properties": {
                "baz": {
                    "type": Foo.DESCRIPTOR,
                },
            },
        },
    ),
)

def write_protos(proto_dir: str):
    """Write out the .proto files for Foo and Bar to the given directory"""
    Foo.write_proto_file(proto_dir)
    Bar.write_proto_file(proto_dir)

Similar Projects

There are a number of similar projects in this space that offer slightly diferent value:

  • jtd-codegen: This project focuses on statically generating language-native code (including python) to represent the JTD schema.
  • py-json-to-proto: This project aims to deduce a schema from an instance of a json object.
  • pure-protobuf: This project has a very similar aim to jtd-to-proto, but it skips the intermediate descriptor representation and thus is not able to produce native message.Message classes.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

jtd_to_proto-0.9.0-py310-none-any.whl (17.2 kB view hashes)

Uploaded Python 3.10

jtd_to_proto-0.9.0-py39-none-any.whl (17.2 kB view hashes)

Uploaded Python 3.9

jtd_to_proto-0.9.0-py38-none-any.whl (17.2 kB view hashes)

Uploaded Python 3.8

jtd_to_proto-0.9.0-py37-none-any.whl (17.2 kB view hashes)

Uploaded Python 3.7

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page