pygrpc-native

A gRPC server for Python that does the HTTP/2 and gRPC work in Rust, outside the GIL.
Your handlers stay in Python.

When a Python gRPC server runs short of CPU, the usual answers are more server processes or a rewrite in another language. pygrpc-native moves the gRPC work itself (HTTP/2, flow control, deadlines, cancellation) into Rust, outside the GIL, and leaves your servicers, interceptors and generated code as they are.

• Switching is one line. pygrpc_native.server() and pygrpc_native.aio.server() take the same arguments as grpcio's servers. Clients don't change.
• It behaves like grpcio. Status codes, metadata, deadlines, cancellation, interceptors and ServicerContext work the same way. The few differences are listed below, and anything else is a bug.
• Nothing stops working. Methods it can't serve itself, such as bidirectional streaming or server reflection, are passed to a grpcio server running behind it.
• It's faster under load. With an empty handler it answers about 1.5x as many calls per second as grpcio from one client thread, and about 5x from 32. See Benchmarks.

Quick start

pip install pygrpc-native

Wheels are available for Linux (x86_64, aarch64) and macOS (arm64, x86_64), for CPython 3.10 and later.

Change the line that creates the server:

 from concurrent import futures
-import grpc
+import pygrpc_native

-server = grpc.server(futures.ThreadPoolExecutor(16))
+server = pygrpc_native.server(futures.ThreadPoolExecutor(16))
 greeter_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
 server.add_insecure_port("[::]:50051")
 server.start()
 server.wait_for_termination()

For grpc.aio:

+import pygrpc_native.aio
+
-server = grpc.aio.server()
+server = pygrpc_native.aio.server()
 greeter_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
 server.add_insecure_port("[::]:50051")
 await server.start()
 await server.wait_for_termination()

To build from source:

pip install maturin
maturin develop --release

How it works

Every call reaches Python as an event: an id, the method, the request bytes and who sent it.

grpc client ──HTTP/2──▶ tokio threads (no GIL)
                          │ path → method id
                          ▼
                    event queue ──▶ recv_batch() in Python
                          ◀── respond(call_id, bytes) from any thread

Python picks up calls in batches, so a burst costs one trip across the GIL instead of one per call. server() and aio.server() are built on top of this and run your handlers the way grpcio does.

What runs natively

The fallback is a grpcio server listening on a Unix socket behind this one. Calls that go there work but aren't any faster, and peer() in them returns the socket. The native server listens on a single insecure port.

Low-level API

NativeGrpcFront is the layer under server(). You receive each call as an event and answer it by id:

import threading
from pygrpc_native import EVENT_CALL, NativeGrpcFront

front = NativeGrpcFront("0.0.0.0:50051", [("/echo.Echo/Say", False)])
front.start()

def serve():
    while (events := front.recv_batch(64)) is not None:
        for kind, call_id, method_id, payload, client_key in events:
            if kind == EVENT_CALL:
                front.respond(call_id, payload)

threading.Thread(target=serve).start()

Streaming, call details, response metadata, bearer tokens and metrics are covered in docs/low-level-api.md.

Benchmarks

An empty unary handler on an Apple M4 Pro, called by grpcio clients on the same machine. Calls per second, median of three runs:

On a 40-vCPU Linux container, the low-level API answered about 12x as many calls as grpcio at 32 threads. The benchmark README has all the numbers and explains how they were measured.

Limitations

• NativeGrpcFront only serves unary and server-streaming methods. Other kinds can go through the fallback server, and server() and aio.server() send them there for you.
• There's no TLS yet. NativeGrpcFront checks bearer tokens itself, while server() and aio.server() run grpcio interceptors.
• As with grpc-core, binding a Unix socket path replaces any socket file already there.
• SO_REUSEPORT is set only when grpc.so_reuseport is 1. grpcio sets it by default, so binding a port that another server already uses works with grpcio but fails here.
• Each connection allows up to 1024 concurrent streams unless grpc.max_concurrent_streams says otherwise. A connection that hasn't finished the HTTP/2 preface within 120 s is dropped.
• maximum_concurrent_rpcs counts a call to a method on the fallback server only while its handler runs, while grpcio also counts calls waiting for a worker. With server(), such a call over the limit is refused when a worker picks it up rather than as it arrives.
• A method that only generic handlers know is looked up when it is first called, so their service() runs once more for that call. A method none of them knows is looked up on each call before it goes to the fallback server.
• Calls sent back to back on one connection can reach handlers out of order. gRPC doesn't guarantee an order across calls, though grpcio usually keeps it. NativeGrpcFront(ordered=True) keeps it.
• A call whose request metadata is over the limit (16 KiB unless set) is refused. Between grpc.max_metadata_size and that limit, grpcio also refuses some calls at random. This server doesn't.

Compatibility

pygrpc-native follows semantic versioning, and the low-level API also stays compatible within 0.x:

• The 0.1 names, signatures and event tuples of NativeGrpcFront won't change or go away. tests/test_api.py pins them. New parameters are only added after the existing ones, with a default.
• The keys that metrics() returns won't be renamed or removed within 0.x, though new ones may be added.
• server() and aio.server() follow grpcio's API. Any difference from grpcio's behavior is a bug unless this README lists it.

Changes are listed in CHANGELOG.md.

Contributing

See CONTRIBUTING.md. Every commit needs a DCO sign-off (git commit -s).

License

Apache-2.0