Skip to main content

Protobuf for humans

Project description

Protox

Protocol buffers implementation for Python 3 that generates human-readable code with type hinting support

Quick example

from enum import IntEnum
from protox import Message
from protox import UInt32, String, EnumField


class User(Message):
    class Role(IntEnum):
        USER = 1
        ADMIN = 2

    id: int = UInt32(number=1, required=True)
    email: str = String(number=2, required=True)
    role: Role = EnumField(Role, number=3, required=True, default=Role.USER)


user = User(id=1, email="john@doe.com", role=User.Role.USER)
user_data = user.to_bytes()

print(User.from_bytes(user_data))
# Outputs:
# message User
#     id = 1
#     email = 'john@doe.com'
#     role = Role.USER

More examples

Code generation

Code generator implemented as a protoc plugin so you have to install it first:

Install protoc (Ubuntu)

sudo apt install protobuf-compiler

Install protoc (Mac OS)

brew install protobuf

Install protox

python3 -m pip install protox

Generate messages

protoc \
    --proto_path=protobuf_src \
    --protox_out=. \
    ./protobuf_src/user.proto

Generate grpclib service with all required dependencies

The following code generates service ping_pong.proto from the protobuf_src directory into {$PROJECT_ROOT}/app/protobuf.

The protox plugin supports 3 options:

  • --base-package=path/to/protobuf/out
  • --grpclib # generates grpclib services
  • --with-dependencies # generates dependencies for the given protobuf file

The options to the plugin are passed using the --protox_opt="{plugin options here}" command

protoc \
    --proto_path=protobuf_src \
    --protox_out=. \
    --protox_opt="--base-package=app/protobuf --grpclib --with-dependencies" \
    ./protobuf_src/ping_pong.proto

Core concepts

  • Human-readable python3.6+ generated code with type hinting
  • Support protobuf 2 and 3 at the same time
  • None values instead of zero values in fields for both proto2 and proto3
  • Message.has_field() in both proto2 and proto3
  • Protocols are easy to describe without code generation
  • Messages implemented in more pythonic way: to_bytes() instead of SerializeToString()
  • Enums are just enums python int enums

Features

  • Messages
  • Enums
  • Nested messages
  • Maps
  • Well-known types
  • Repeated fields
  • Repeated messages
  • Repeated enums
  • Custom Message.to_python() / from_python() functions
  • Group fields (Deprecated by protobuf)

Code generator features

  • Protobuf
  • Grpclib
  • Grpc.io
  • Custom python package for protobuf out messages
  • Compile protobuf file with dependencies
  • Names mangling to avoid reserved names collisions
  • Recursive messages/enums support
  • Field names to_snake_case support

Difference with google's protobuf implementation

Binary protocol works exactly as google's implementation does.

The difference is in the way messages behave:

  • Fields that were not explicitly set are None rather than zero-values
  • Methods like SerializeToString() were changed to more pythonic alternatives like to_bytes() / from_bytes()
  • Enums are just python int enums

Generated code example

Grpclib service example

from enum import IntEnum

import protox


class User(protox.Message):
    class Status(IntEnum):
        USER = 1
        ADMIN = 2

    id: int
    email: str
    status: 'User.Status'

    def __init__(
        self,
        *,
        id: int = None,
        email: str = None,
        status: 'User.Status' = None,
    ):
        super().__init__(
            id=id,
            email=email,
            status=status,
        )


protox.define_fields(
    User,
    id=protox.Int32(
        number=1, required=True
    ),
    email=protox.String(
        number=2, required=True
    ),
    status=protox.EnumField(
        number=3, py_enum=User.Status, default=User.Status.USER, required=True
    ),
)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

protox-0.4.3.tar.gz (30.9 kB view details)

Uploaded Source

File details

Details for the file protox-0.4.3.tar.gz.

File metadata

  • Download URL: protox-0.4.3.tar.gz
  • Upload date:
  • Size: 30.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.52.0 CPython/3.8.10

File hashes

Hashes for protox-0.4.3.tar.gz
Algorithm Hash digest
SHA256 44f6035a29000bc4dc06cc5276a6be814ef758e88955a2dd740f71d362b82567
MD5 249cb9171bd124f45027db0178a77f2f
BLAKE2b-256 8ab7f713abfa90878683e5d53343f5c5cae7c9bd6787df260713722eaec7190d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page