Skip to main content

Python protocol buffers for the rest of us

Project description

Protocol Buffers for the Rest of Us



Are you annoyed by having to fix the absolute imports generated by protoc?

If so, then protoletariat is the tool for you.

protoletariat has one goal: fixing the broken imports for the Python code generated by protoc.

See for the discussion that inspired this tool.


Artifact Status Installation Command
PyPI Package PyPI pip install protoletariat
Conda Package Conda Version conda install protoletariat -c conda-forge
Docker Image docker pull

Note: the conda-forge package version may lag behind the other artifacts by a few hours.


protoletariat is designed to be run as a post-processing step after running protoc. It operates directly on the generated code.


Here's an example of how to use the tool, called protol:

  1. Create a few protobuf files
// thing1.proto
syntax = "proto3";

import "thing2.proto";

package things;

message Thing1 {
  Thing2 thing2 = 1;
// thing2.proto
syntax = "proto3";

package things;

message Thing2 {
  string data = 1;
  1. Run protoc on those files
$ mkdir out
$ protoc \
  --python_out=out \
  --proto_path=directory/containing/protos thing1.proto thing2.proto
  1. Run protol on the generated code
$ protol \
  --create-package \
  --in-place \
  --python-out out \
  protoc --proto-path=directory/containing/protos thing1.proto thing2.proto

The out/ module should show a diff containing at least these lines:

-import thing2_pb2 as thing2__pb2
+from . import thing2_pb2 as thing2__pb2

How it works

At a high level protoletariat converts absolute imports to relative imports.

However, it doesn't convert just any absolute import to a relative import.

The protol tool will only convert imports that were generated from .proto files. It does this by inspecting FileDescriptorProtos generated from the proto files.

The core rewrite mechanism is implemented using a simplified form of pattern matching that looks at the Python AST, and invokes rewrite rules for matched import patterns.


protoletariat has a subcommand for each tool that you might like to use to generate FileDescriptorSet bytes:

Subcommand Description
protoc Uses protoc to generate FileDescriptorSet bytes
buf Uses buf to generate FileDescriptorSet bytes
raw You provide the FileDescriptorSet bytes as a file or directly from stdin


$ protol
Usage: protol [OPTIONS] COMMAND [ARGS]...

  Rewrite protoc or buf-generated imports for use by the protoletariat.

  -o, --python-out DIRECTORY      Directory containing protoc or buf-generated Python code  [required]
  --in-place / --not-in-place     Overwrite all relevant files under `--python-out` with adjusted imports  [default: not-in-place]
  --create-package / --dont-create-package
                                  Recursively create files under `--python-out`  [default: dont-create-package]
  -s, --module-suffixes TEXT      Suffixes of Python/mypy modules to process  [default:, _pb2.pyi,, _pb2_grpc.pyi]
  --exclude-google-imports / --dont-exclude-google-imports
                                  Exclude rewriting imports prefixed with google/protobuf
  -e, --exclude-imports-glob TEXT
                                  Exclude imports matching a glob pattern from being rewritten. Multiple values are allowed
  --help                          Show this message and exit.

  buf     Use buf to generate the FileDescriptorSet blob
  protoc  Use protoc to generate the FileDescriptorSet blob
  raw     Rewrite imports using FileDescriptorSet bytes from a file or stdin

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

protoletariat-3.2.19.tar.gz (19.7 kB view hashes)

Uploaded Source

Built Distribution

protoletariat-3.2.19-py3-none-any.whl (20.4 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page