pygrpc-native

A gRPC server for Python whose HTTP/2 and gRPC work runs in Rust, outside the GIL. Your handlers stay in Python.

When a Python gRPC server runs out of CPU, the usual fixes are running more server processes or rewriting the servicers in another language. pygrpc-native instead moves the gRPC work itself (framing, flow control, deadlines and cancellation) onto a tokio runtime outside the GIL, and leaves your servicers, interceptors and generated code alone, so Python spends its time on your code. pygrpc_native.server() and pygrpc_native.aio.server() take the same arguments as grpcio's servers, so switching is usually a one-line change, and nothing changes for clients.

Underneath, each call reaches Python as an event: an id, the method, the request bytes and who sent it.

grpc client ──HTTP/2──▶ tokio threads (no GIL)
                          │ path → method id
                          ▼
                    event queue ──▶ recv_batch() in Python
                          ◀── respond(call_id, bytes) from any thread

Calls reach Python in batches, so a burst takes one trip across the GIL rather than one per call.

Install

pip install pygrpc-native

Wheels are published for Linux (x86_64, aarch64) and macOS (arm64, x86_64) and work on CPython 3.10 and later. To build from a checkout:

pip install maturin
maturin develop --release

With grpcio servicers

pygrpc_native.server() takes the same arguments as grpc.server() and serves servicers from grpcio-tools generated code (1.64 or later) unchanged:

from concurrent import futures
import pygrpc_native

server = pygrpc_native.server(futures.ThreadPoolExecutor(16))
greeter_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
server.add_insecure_port("[::]:50051")
server.start()
server.wait_for_termination()

Status codes, error messages, metadata, deadlines, cancellation and ServicerContext behave as they do in grpcio.

Unary and server-streaming methods are served natively on one insecure port. The rest, such as client-streaming and bidirectional methods, server reflection, or handlers added only with add_generic_rpc_handlers, as code generated by grpcio-tools before 1.64 does, go to a grpcio server on a Unix socket behind it. They work, but they aren't any faster, and peer() in them is the socket.

Interceptors run as in grpcio, on the methods served here and on the fallback server. Of the server options, the message size limits, the keepalive settings (grpc.keepalive_time_ms, grpc.keepalive_timeout_ms and grpc.keepalive_permit_without_calls), grpc.max_concurrent_streams and the metadata size limits (grpc.max_metadata_size and grpc.absolute_max_metadata_size) work as in grpcio. Options that ask for something this server can't do raise an error: closing connections by age or idleness (grpc.max_connection_age_ms, grpc.max_connection_age_grace_ms, grpc.max_connection_idle_ms), grpc.max_allowed_incoming_connections, response compression and grpc.so_reuseport, as do maximum_concurrent_rpcs and TLS. Any other option is ignored, as grpcio ignores the many that only clients read. That covers the limits on client pings, which are accepted but not enforced, and grpc.http2.max_frame_size, which grpc-core's flow control overrides anyway. Values that aren't integers are ignored too, as in grpcio.

There are two known differences. If a unary handler calls send_initial_metadata() and then fails, the metadata goes out with the status, so the client sees it as trailing metadata. And a connection whose client doesn't answer a keepalive ping is closed after grpc.keepalive_timeout_ms, as gRPC's keepalive documentation describes, while grpcio waits for grpc.http2.ping_timeout_ms, a minute by default.

pygrpc_native.aio.server() does the same for grpc.aio.server():

import pygrpc_native.aio

server = pygrpc_native.aio.server()
greeter_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
server.add_insecure_port("[::]:50051")
await server.start()
await server.wait_for_termination()

Handlers run on the event loop that called start(), uvloop included. abort() raises grpc.aio.AbortError, and a handler whose client cancels or whose deadline passes gets asyncio.CancelledError, as in grpc.aio. Streams can yield from an async generator or call context.write(), and handlers that aren't coroutines run in migration_thread_pool with grpcio's sync context, where abort() returns instead of raising. The fallback, the unsupported options and the known differences are the same as for the sync server.

Low-level API

NativeGrpcFront is the layer under server(). Each call reaches Python as an event that you answer by id:

import threading
from pygrpc_native import EVENT_CALL, NativeGrpcFront

front = NativeGrpcFront("0.0.0.0:50051", [("/echo.Echo/Say", False)])
front.start()

def serve():
    while (events := front.recv_batch(64)) is not None:
        for kind, call_id, method_id, payload, client_key in events:
            if kind == EVENT_CALL:
                front.respond(call_id, payload)

threading.Thread(target=serve).start()

Streaming, call details, response metadata, bearer tokens and metrics are covered in docs/low-level-api.md.

Compatibility

pygrpc-native follows semantic versioning, and the low-level API stays compatible within 0.x as well. tests/test_api.py pins the 0.1 names, signatures and event tuples of NativeGrpcFront: they won't change or go away, and new parameters only come after the existing ones, with a default. The keys metrics() returns won't be renamed or removed within 0.x either, though new ones may be added. server() and aio.server() follow grpcio's API, and any difference from grpcio's behavior is a bug unless it is listed in this README. Changes are listed in CHANGELOG.md.

Limitations

• NativeGrpcFront serves unary and server-streaming methods only; other kinds can go through the fallback server. server() and aio.server() send them there on their own.
• No TLS yet. NativeGrpcFront checks bearer tokens itself; server() and aio.server() run grpcio interceptors instead.
• Like grpc-core, binding a Unix socket path replaces any socket file already there.
• The server never sets SO_REUSEPORT, so binding a port another server is using fails, where grpcio's default lets both bind it. grpc.so_reuseport set to 0 is accepted; other integers raise an error.
• Per connection: at most 1024 concurrent streams unless grpc.max_concurrent_streams says otherwise. A connection that hasn't finished the HTTP/2 preface after 120 s is dropped.
• A call with more request metadata than the limit (16 KiB unless set) is refused. grpcio also refuses some calls at random between grpc.max_metadata_size and that limit; this server doesn't.

Benchmarks

With an empty handler on an Apple M4 Pro, pygrpc_native.server() answers about 1.5x the calls per second of grpc.server() from one client thread and about 5x from 32. On a 40-vCPU Linux container the low-level API answered about 12x at 32 threads. See benchmarks for the numbers and how they were measured.

Contributing

See CONTRIBUTING.md. Every commit needs a DCO sign-off (git commit -s).

License

Apache-2.0