magicproto

Import .proto files directly. No protoc, no generated _pb2.py files, no build step:

import magicproto                       # installs the import hook
from magicproto.greet import hello_pb2, hello_pb2_grpc

req = hello_pb2.HelloRequest(name="world")
stub = hello_pb2_grpc.GreeterStub(channel)

magicproto.greet.hello_pb2 compiles greet/hello.proto (found on MAGICPROTO_PATH / sys.path) at import time and synthesizes the module. The dotted Python path mirrors the canonical proto path 1:1, so the import name, the file location, and the descriptor name can never drift apart — the classic grpc_tools "the generated import points at the wrong place" problem cannot occur.

How it works

1. A scoped sys.meta_path finder claims names under magicproto. (only).
2. magicproto.greet.hello_pb2 → canonical greet/hello.proto, located on the include roots (the protoc -I model: MAGICPROTO_PATH then sys.path).
3. The Rust extension (magicproto._compiler, built on the pure-Rust protox compiler) turns the .proto into a serialized FileDescriptorSet — the one thing the stock protobuf runtime can't do itself.
4. Those descriptors are registered through the stock descriptor_pool.Default() + google.protobuf.internal.builder, i.e. the exact path a generated _pb2.py uses, so the message classes are indistinguishable from generated ones.
5. _pb2_grpc modules are synthesized directly from the service descriptors.

See DESIGN.md for the full rationale, the multi-package namespacing model, and the limitations.

Where to put protos

Put foo/bar.proto where you'd want foo/bar.py, and import it as magicproto.foo.bar_pb2.

A library ships its protos as package data inside its own package directory; the directory name namespaces them, so two installed libraries can't collide.

Build (development)

python -m venv --system-site-packages .venv
source .venv/bin/activate
maturin develop
python examples/demo.py
pytest

Status

Prototype. Reuses the stock protobuf runtime for messages; a production build would route descriptor construction through a dedicated upb-backed runtime instead. See DESIGN.md § Limitations.