Protocol Buffers for the Rest of Us
Motivation
Are you annoyed by having to fix the absolute imports generated by protoc?
If so, then protoletariat is the tool for you.
protoletariat has one goal: fixing the broken imports for the Python code
generated by protoc.
See https://github.com/protocolbuffers/protobuf/issues/1491 for the discussion that inspired this tool.
Installation
| Artifact | Status | Installation Command |
|---|---|---|
| PyPI Package | pip install protoletariat |
|
| Conda Package | conda install protoletariat -c conda-forge |
|
| Docker Image | ∅ | docker pull ghcr.io/cpcloud/protoletariat:latest |
Note: the conda-forge package version may lag behind the other artifacts by a few hours.
Usage
protoletariat is designed to be run as a post-processing step after running
protoc. It operates directly on the generated code.
Example
Here's an example of how to use the tool, called protol:
- Create a few protobuf files
// thing1.proto
syntax = "proto3";
import "thing2.proto";
package things;
message Thing1 {
Thing2 thing2 = 1;
}
// thing2.proto
syntax = "proto3";
package things;
message Thing2 {
string data = 1;
}
- Run
protocon those files
$ mkdir out
$ protoc \
--python_out=out \
--proto_path=directory/containing/protos thing1.proto thing2.proto
- Run
protolon the generated code
$ protol \
--create-package \
--in-place \
--python-out out \
protoc --proto-path=directory/containing/protos thing1.proto thing2.proto
The out/thing1_pb2.py module should show a diff containing at least these lines:
-import thing2_pb2 as thing2__pb2
-
+from . import thing2_pb2 as thing2__pb2
How it works
At a high level protoletariat converts absolute imports to relative imports.
However, it doesn't convert just any absolute import to a relative import.
The protol tool will only convert imports that were generated from .proto
files. It does this by inspecting FileDescriptorProtos generated from the
proto files.
The core rewrite mechanism is implemented using a simplified form of pattern matching that looks at the Python AST, and invokes rewrite rules for matched import patterns.
Subcommands
protoletariat has a subcommand for each tool that you might like to use to
generate FileDescriptorSet bytes:
| Subcommand | Description |
|---|---|
protoc |
Uses protoc to generate FileDescriptorSet bytes |
buf |
Uses buf to generate FileDescriptorSet bytes |
raw |
You provide the FileDescriptorSet bytes as a file or directly from stdin |
Help
$ protol
Usage: protol [OPTIONS] COMMAND [ARGS]...
Rewrite protoc or buf-generated imports for use by the protoletariat.
Options:
-o, --python-out DIRECTORY Directory containing protoc or buf-generated Python code [required]
--in-place / --not-in-place Overwrite all relevant files under `--python-out` with adjusted imports [default: not-in-place]
--create-package / --dont-create-package
Recursively create __init__.py files under `--python-out` [default: dont-create-package]
-s, --module-suffixes TEXT Suffixes of Python/mypy modules to process [default: _pb2.py, _pb2.pyi, _pb2_grpc.py, _pb2_grpc.pyi]
--exclude-google-imports / --dont-exclude-google-imports
Exclude rewriting imports prefixed with google/protobuf
-e, --exclude-imports-glob TEXT
Exclude imports matching a glob pattern from being rewritten. Multiple values are allowed
--help Show this message and exit.
Commands:
buf Use buf to generate the FileDescriptorSet blob
protoc Use protoc to generate the FileDescriptorSet blob
raw Rewrite imports using FileDescriptorSet bytes from a file or stdin
Metadata
Release files for protoletariat 3.3.10
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| protoletariat-3.3.10.tar.gz | 20.7 kB | Without provenance | Details for protoletariat-3.3.10.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| protoletariat-3.3.10-py3-none-any.whl | Python 3 | none | any | Details for protoletariat-3.3.10-py3-none-any.whl |
Total release size: 41.4 kB
Release files / protoletariat-3.3.10.tar.gz
| Download URL | protoletariat-3.3.10.tar.gz |
|---|---|
| Size | 20.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
bfa0e663fe4c9e3584101779b8b9f5a77b8af623aa25f680e0559b7d2609039b
|
|
BLAKE2b-256 checksum How to use checksums |
366031c8dac4e060bce3f102195409128838fc5830e4e589f7962eb7531b4a27
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.5 CPython/3.12.8 Linux/6.8.0-1021-azure
|
Release files / protoletariat-3.3.10-py3-none-any.whl
| Download URL | protoletariat-3.3.10-py3-none-any.whl |
|---|---|
| Size | 20.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
4b0ab790a1791613d9ef1c4159e03424a7f056592ca66578ba585ae96686ee2d
|
|
BLAKE2b-256 checksum How to use checksums |
306bd8b8fb09a31f85685d41f63d0d4a807c14aca175c71e17bebcfbf5dc768a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.5 CPython/3.12.8 Linux/6.8.0-1021-azure
|
Security for protoletariat 3.3.10
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.