proto

Protocol Buffers without the toolchain. proto lets you declare binary messages as ordinary Python dataclasses and serialize them to bytes that are wire-compatible with Google's Protocol Buffers (proto3). No protoc, no generated code, no C extension, no dependencies: just the standard library.

Use it when you need to talk to a protobuf service from a script, persist compact records, or prototype a schema in Python first, and still have every other protobuf implementation read your bytes.

import proto

@proto.message
class Point:
    x: int = proto.field(1, "sint32")
    y: int = proto.field(2, "sint32")

data = proto.encode(Point(3, -4))     # b'\x08\x06\x10\x07'
assert proto.decode(Point, data) == Point(3, -4)

Features

• Wire-compatible: varints, zigzag, fixed-width, length-delimited and packed encodings match the official protobuf encoding byte for byte. The tests check the exact byte strings from the protobuf encoding guide.
• Schema-first dataclasses: field numbers live next to the type annotations. Messages are real dataclasses, so you keep ==, repr, replace() and your type checker.
• All 15 scalar types: int32 int64 uint32 uint64 sint32 sint64 bool fixed32 fixed64 sfixed32 sfixed64 float double string bytes, plus IntEnum enums, nested and recursive messages, repeated fields, and proto3 optional.
• proto3 semantics: default values are omitted on the wire, unknown fields are skipped, packed and unpacked repeated fields are both accepted, unknown enum values are kept as int (open enums).
• Strict validation: out-of-range integers, wrong Python types, malformed or truncated input raise clear EncodeError / DecodeError / SchemaError.
• .proto export: to_proto() renders your classes as a .proto file, so other languages can generate code from the same schema.
• Streaming: length-delimited framing (writeDelimitedTo format) for many messages in one file or socket.
• Pure Python 3.10+, fully type-hinted (py.typed), zero dependencies.

Install

pip install proto            # once published
pip install -e .             # from a checkout

Note: Google's proto-plus package also installs a top-level proto module. Don't install both in the same environment.

Quickstart

from __future__ import annotations

import enum
import io

import proto


class Role(enum.IntEnum):
    ROLE_UNSPECIFIED = 0     # proto3 enums need a zero value
    ADMIN = 1
    MEMBER = 2


@proto.message
class Address:
    city: str = proto.field(1)
    zip_code: str = proto.field(2)


@proto.message
class User:
    name: str = proto.field(1)                      # inferred: string
    id: int = proto.field(2, "uint32")              # explicit scalar type
    age: int | None = proto.field(3, "int32")       # proto3 `optional`: None = unset
    role: Role = proto.field(4)                     # enum
    emails: list[str] = proto.field(5)              # repeated
    scores: list[int] = proto.field(6, "sint32")    # repeated, packed by default
    address: Address | None = proto.field(7)        # nested message
    friends: list[User] = proto.field(8)            # recursive


ada = User("ada", id=1, role=Role.ADMIN, emails=["ada@example.com"],
           address=Address("London", "N1"))

data = ada.to_bytes()                 # same as proto.encode(ada)
again = User.from_bytes(data)         # same as proto.decode(User, data)
assert again == ada

proto.to_dict(ada)
# {'name': 'ada', 'id': 1, 'role': 'ADMIN', 'emails': ['ada@example.com'],
#  'scores': [], 'address': {'city': 'London', 'zip_code': 'N1'}, 'friends': []}

# Many messages in one stream
buf = io.BytesIO()
for user in (ada, User("bob", id=2)):
    proto.write_delimited(buf, user)
buf.seek(0)
names = [u.name for u in proto.iter_delimited(User, buf)]   # ['ada', 'bob']

print(proto.to_proto(User, package="example.v1"))

The last line prints:

syntax = "proto3";

package example.v1;

enum Role {
  ROLE_UNSPECIFIED = 0;
  ADMIN = 1;
  MEMBER = 2;
}

message Address {
  string city = 1;
  string zip_code = 2;
}

message User {
  string name = 1;
  uint32 id = 2;
  optional int32 age = 3;
  Role role = 4;
  repeated string emails = 5;
  repeated sint32 scores = 6;
  Address address = 7;
  repeated User friends = 8;
}

Declaring fields

Pass type= (the second argument of field) to choose another scalar encoding such as "sint32" or "fixed64", or to name an enum/message class explicitly.

API overview

Everything is importable from the top-level proto package.

Low-level primitives live in proto.wire:

Limitations

proto 0.1 covers the core of proto3. Not supported yet: oneof, map<K, V> fields, well-known types (Timestamp, Any, ...), proto2 groups (they are skipped when decoding), services/gRPC, and merging of repeated occurrences of a singular message field (the last occurrence wins). Annotations that name other classes must be resolvable from the scope where the class is defined.

Development

PYTHONPATH=src python3 -m unittest discover -s tests -v   # standard library only
python3 -m pytest                                        # if pytest is installed

License

MIT