pygrpc-native

A gRPC server for Python whose HTTP/2 and gRPC work runs in Rust, outside the GIL. Your handlers stay in Python.

grpcio runs its own Python code for every call, under the same GIL as your handlers. pygrpc-native does the framing, flow control, deadlines and cancellation on a tokio runtime instead, and gives Python each call as an event: an id, the method, the request bytes and who sent it. You decode the protobuf, run the handler and send back the encoded response. Clients and the wire format are the same as with grpcio.

grpc client ──HTTP/2──▶ tokio threads (no GIL)
                          │ path → method id
                          ▼
                    event queue ──▶ recv_batch() in Python
                          ◀── respond(call_id, bytes) from any thread

Calls reach Python in batches, so a burst takes one trip across the GIL rather than one per call.

Install

pip install pygrpc-native

Wheels are published for Linux (x86_64, aarch64) and macOS (arm64, x86_64) and work on CPython 3.10 and later. To build from a checkout:

pip install maturin
maturin develop --release

With grpcio servicers

pygrpc_native.server() takes the same arguments as grpc.server() and serves servicers from grpcio-tools generated code (1.62 or later) unchanged:

from concurrent import futures
import pygrpc_native

server = pygrpc_native.server(futures.ThreadPoolExecutor(16))
greeter_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
server.add_insecure_port("[::]:50051")
server.start()
server.wait_for_termination()

Status codes, error messages, metadata, deadlines, cancellation and ServicerContext behave as they do in grpcio. Unary and server-streaming methods are served natively on one insecure port. Everything else, such as client-streaming and bidirectional methods, server reflection or handlers added only with add_generic_rpc_handlers, is passed to a grpcio server on a Unix socket behind it, so it works but doesn't get faster, and peer() there is the socket. Interceptors, maximum_concurrent_rpcs, TLS, response compression and server options other than the message size limits and client ping settings raise an error instead of being ignored. One difference remains: a unary handler that calls send_initial_metadata() and then fails sends the metadata with the status instead of before it, so the client sees it as trailing metadata.

pygrpc_native.aio.server() does the same for grpc.aio.server():

import pygrpc_native.aio

server = pygrpc_native.aio.server()
greeter_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
server.add_insecure_port("[::]:50051")
await server.start()
await server.wait_for_termination()

Handlers run on the event loop that called start(), uvloop included. abort() raises grpc.aio.AbortError, and a handler whose client cancels or whose deadline passes gets asyncio.CancelledError, as in grpc.aio. Streams can yield from an async generator or call context.write(), and handlers that aren't coroutines run in migration_thread_pool. The fallback, the unsupported options and the unary metadata difference are the same as for the sync server.

Low-level example

import threading
from pygrpc_native import EVENT_CALL, NativeGrpcFront

front = NativeGrpcFront("0.0.0.0:50051", [("/echo.Echo/Say", False)])
front.start()

def serve():
    while (events := front.recv_batch(64)) is not None:
        for kind, call_id, method_id, payload, client_key in events:
            if kind == EVENT_CALL:
                front.respond(call_id, payload)

threading.Thread(target=serve).start()

Any grpcio client can call it:

import grpc

with grpc.insecure_channel("localhost:50051") as channel:
    print(channel.unary_unary("/echo.Echo/Say")(b"hello"))

In a real server payload is a serialized protobuf message: decode it with the generated class, run the handler (in a thread pool, if it blocks) and pass the serialized reply to respond.

API

NativeGrpcFront(target, methods, io_threads=2, client_key_metadata=None, call_details=False, delay_stream_headers=False, bearer_token_sha256=None, unauthenticated_message="Unauthenticated") binds host:port (every address the host resolves to), unix:path or unix:///path. Port 0 picks a free port. methods lists (path, server_streaming) pairs, where a path is /package.Service/Method and a method's id is its index in the list. add_methods appends more before start(), so the port can be bound before the methods are known.

recv_batch returns (kind, call_id, method_id, payload, client_key) tuples. kind is EVENT_CALL for a new call and EVENT_CANCEL when the client left a stream before it ended (payload and client_key are then None). client_key is the value of the client_key_metadata request metadata when it is set and present, and the peer address otherwise, so calls from the same client can be routed to the same worker.

With call_details=True, recv_batch_details adds (metadata, peer, time_remaining) to each call, matching grpcio's invocation_metadata(), peer() and time_remaining(): the request metadata in order without the transport headers, with -bin values as bytes, the peer as ipv4:host:port, and the seconds left before the deadline or None. Without it the sixth item is None, and nothing extra is collected per call. The option also sends EVENT_CANCEL for a unary call that ends before it is answered, as grpcio lets a handler see through is_active().

A stream sends its response headers as soon as the call starts. With delay_stream_headers=True they wait, as in grpcio, for the first message or stream_send_initial_metadata, which is how a stream gets initial metadata; a stream that ends before either is answered with trailers only.

Response metadata is a sequence of (key, value) pairs, with bytes values for -bin keys and str otherwise. Keys must be lowercase and can't start with grpc-, except that fail accepts grpc-status-details-bin for a rich status, as grpcio's abort_with_status sends it.

With bearer_token_sha256, a list of SHA-256 digests of the accepted tokens, every call must send authorization: Bearer <token> for one of them. Other calls end with UNAUTHENTICATED and unauthenticated_message on the tokio side, before the method lookup, so they never reach Python or the fallback server. The scheme is matched in any case, and the server only keeps the digests:

import hashlib

front = NativeGrpcFront(
    "0.0.0.0:50051",
    methods,
    bearer_token_sha256=[hashlib.sha256(key.encode()).digest() for key in keys],
)

set_fallback lets one port serve methods you don't list: run a regular grpc.Server on a Unix socket and its methods are proxied there, with its interceptors.

Limitations

• Unary and server-streaming methods only. Other kinds can go through the fallback server.
• No TLS yet, and grpcio interceptors don't run on the listed methods. Bearer tokens are the only authentication, and only on NativeGrpcFront.
• Clients of a Unix socket server must set grpc.default_authority, since the socket path grpc-core sends as :authority is not valid HTTP/2.
• Like grpc-core, binding a Unix socket path replaces any socket file already there.
• Per connection: at most 1024 concurrent streams and 16 KiB of request metadata. A connection that hasn't finished the HTTP/2 preface after 120 s is dropped.

Benchmarks

With an empty handler, pygrpc-native answers about 1.6x the calls per second of grpcio's servers from one client thread, and 4x to 12x from 16 or 32. See benchmarks.

Tests

cargo test --locked --no-default-features
maturin develop --release
pytest tests

Contributing

See CONTRIBUTING.md. Every commit needs a DCO sign-off (git commit -s).

License

Apache-2.0