pyipcnode
Peer-to-peer method calls between Python processes on the same machine. Processes join a named system, register methods, look up peers, and call them over shared memory. There is no server process.
Calls go through a shared directory plus per-node shared-memory mailboxes, not sockets or pipes. That keeps same-machine RPC cheap:
- No broker. Any node can call any other once it knows
(node_name, pid). Nothing sits in the middle, and nothing has to stay up as a server. - Low latency. A call is a write into the peer's mailbox and a wake-up. There is no TCP/loopback stack, no listen port, and no extra process hop.
- Local discovery. Peers in the same
system_nameshow up in a shared table.list_nodes()is a local read; you do not poll a name service. - Compact payloads. Arguments travel as little-endian binary, not JSON text over a socket. Large arrays stay packed (
u32count + tightly packed elements). - Isolation by name. Different
system_namevalues are separate systems on the same machine. Processes that should not talk simply do not share a name.
This is for same-OS, same-machine processes. It is not a network RPC library.
Install
pip install pyipcnode
Requires Python 3.14. Nodes must run on the same OS and the same machine, and they must use the same system_name.
Quick start
Two processes, same system_name. One registers add; the other finds it and calls it.
Callee
from pyipcnode import IpcNode, i4
def add(a: i4, b: i4) -> i4:
return a + b
with IpcNode("worker", "mysystem", "box_worker", 64 * 1024) as node:
node.register("add", add)
input("running, press Enter to exit\n")
Caller
from pyipcnode import (
STATUS_OK,
IpcNode,
decode_result,
encode_args,
find_method,
parse_list_methods,
)
with IpcNode("client", "mysystem", "box_client", 64 * 1024) as node:
peer = next(n for n in node.list_nodes() if n.node_name == "worker")
listed = node.call_raw(peer.node_name, peer.pid, "sys.list_methods", b"", 1000).result()
method = find_method(parse_list_methods(listed.result), "add")
result = node.call_raw(
peer.node_name, peer.pid, "add", encode_args(method, 2, 3), 1000
).result()
if result.status == STATUS_OK:
print(decode_result(method["return"], result.result)) # 5
Create a node
IpcNode(node_name, system_name, recv_shm_name, recv_shm_bytes)
| Argument | Meaning |
|---|---|
node_name |
Logical name of this node. Duplicates are allowed; identity is (node_name, pid). |
system_name |
Communication system. Required, non-empty, [A-Za-z0-9_] only. Nodes with different names cannot see each other. |
recv_shm_name |
Base name of this node's receive mailbox. The library appends _ and the process pid. |
recv_shm_bytes |
Mailbox size in bytes. Required; no default. |
node.pid is read-only and taken from the current process. Construction failures (bad names, mailbox too small, system table full, and similar) raise RuntimeError.
Always close the node. A live node holds a slot in the system table:
with IpcNode("worker", "mysystem", "box_worker", 64 * 1024) as node:
...
node.close() # safe to call more than once
After close(), register, call_raw, and list_nodes raise RuntimeError.
Register methods
Prefer type annotations. Every parameter and the return type must be annotated with a marker from this package. Bare int, str, or float is rejected.
from pyipcnode import IpcNode, char_p, i4
def add(a: i4, b: i4) -> i4:
return a + b
def echo(text: char_p) -> char_p:
return text
def ping() -> None:
return None
def sum_vec(xs: list[i4]) -> i4:
return sum(xs)
node.register("add", add) # recommended: explicit name
node.register(ping) # uses ping.__name__
register returns True on success. Name clashes, empty names, and names starting with sys. return False. Missing or unsupported annotations raise TypeError and nothing is registered.
Built-in methods sys.list_methods and sys.list_nodes are registered at startup and cannot be overridden.
Annotation markers
| Annotation | Wire type | Python value |
|---|---|---|
i1 i2 i4 i8 |
signed integers | int (range-checked) |
u1 u2 u4 u8 |
unsigned integers | int (range-checked) |
f4 f8 |
IEEE-754 float / double | float |
char_p |
UTF-8 string | str |
list[i4] (and other integer/float markers) |
variable-length array | list |
list elements must be integer or float markers. Nested lists and list[char_p] are not supported. bool is not a wire type; True is not accepted as an integer. A typed method returns nothing (-> None) or exactly one value.
Call a peer
call_raw is the only send API. It returns a concurrent.futures.Future immediately. The IPC timeout is timeout_ms (required, milliseconds, >= 0). Failures are reported on CallResult.status; they do not raise.
future = node.call_raw(node_name, pid, method, args, timeout_ms)
cr = future.result() # wait until the IPC finishes
cr = future.result(timeout=2) # extra Python-side wait; does not cancel the call
Future.cancel() does not cancel an in-flight call and returns False.
Discover peers and methods
peers = node.list_nodes() # local snapshot: list[NodeInfo(node_name, pid)]
list_nodes() reads the local system table. It does not probe whether a peer is still alive.
To inspect a peer's methods:
cr = node.call_raw(peer.node_name, peer.pid, "sys.list_methods", b"", 1000).result()
methods = parse_list_methods(cr.result)
spec = find_method(methods, "add")
find_method returns None when the name is missing.
Pack arguments and unpack results
Use the peer's method description (from sys.list_methods, or a dict you already have):
args = encode_args(spec, 2, 3)
cr = node.call_raw(peer.node_name, peer.pid, "add", args, 1000).result()
if cr.status == STATUS_OK:
value = decode_result(spec["return"], cr.result)
encode_args accepts either the full method dict or its params list. Values must match the contract in count and order.
| Contract | Accepted values |
|---|---|
| integers | int (not bool), in range |
f4 / f8 |
int or float |
char* |
str |
short T* |
list of scalars |
large T* |
array.array with a matching typecode, or bytes / bytearray / memoryview |
decode_result handles empty, scalar, and string returns. Array returns (T*) must use decode_array: u1* becomes bytes, other element types become array.array.
import array
args = encode_args(spec, array.array("i", [1, 2, 3]))
cr = node.call_raw(peer.node_name, peer.pid, "sum_vec", args, 1000).result()
Status codes
Future.result() always returns a CallResult unless the extension itself fails. Check status; do not treat a non-zero status as an exception.
| Constant | Value | Meaning |
|---|---|---|
STATUS_OK |
0 | Success. result holds the return payload. |
STATUS_TIMEOUT |
1 | No reply before timeout_ms. |
STATUS_EXEC_FAILED |
2 | Method missing, handler raised, peer gone, or unpack failed on the callee. |
STATUS_DELIVER_FAILED |
3 | Peer's mailbox is full. Retry yourself; the library does not. |
On failure, CallResult.result is b"".
Raw bytes API
Custom layouts, multiple return values, or types outside the marker table stay on the bytes path. call_raw still takes and returns raw bytes.
import struct
def add_fn(args: bytes) -> bytes:
a, b = struct.unpack_from("<ii", args, 0)
return struct.pack("<i", a + b)
node.register(
{
"type": "method",
"name": "add",
"params": [{"type": "i4"}, {"type": "i4"}],
"return": [{"type": "i4"}],
},
add_fn,
)
cr = node.call_raw(peer.node_name, peer.pid, "add", struct.pack("<ii", 1, 2), 1000).result()
Typed register and raw register(dict, fn) can be mixed on the same node.
Threading
Registered functions run on a background thread (the library acquires the GIL). Do not call Future.result() on a same-node in-flight call from inside a registered function; that can deadlock. Waiting from another thread is fine.
Two nodes must be able to call each other at the same time: while one side waits on result(), the other side's registered functions must still run.
C++
The installed package includes the C++ sources at pyipcnode/ipcnode (CMakeLists.txt, include/, src/). Build that directory with your own compiler and C++ standard (C++20 or newer). Pass the directory that contains BoostConfig.cmake as Boost_DIR. The library only calls find_package(Boost); Asio and JSON are header-only.
# IPCNODE_DIR = .../site-packages/pyipcnode/ipcnode
set(Boost_DIR "C:/path/to/lib/cmake/Boost" CACHE PATH "")
FetchContent_Declare(ipcnode SOURCE_DIR "${IPCNODE_DIR}")
FetchContent_MakeAvailable(ipcnode)
target_link_libraries(your_target PRIVATE ipcnode)
python -c "import pyipcnode, pathlib; print(pathlib.Path(pyipcnode.__file__).resolve().parent / 'ipcnode')"
Notes
- Integers and floats on the wire are little-endian.
char*is UTF-8 with a trailing NUL; the length prefix includes that byte.T*is a little-endianu32element count, then packed elements.- The library does not auto-fetch or cache a peer's method list.
__del__callsclose()as a fallback. Do not rely on garbage collection to release the node.
Metadata
Release files for pyipcnode 0.1.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| pyipcnode-0.1.4.tar.gz | 292.9 kB | Without provenance | Details for pyipcnode-0.1.4.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pyipcnode-0.1.4-cp314-cp314-win_amd64.whl | CPython 3.14 | CPython 3.14 | Windows x86-64 | Details for pyipcnode-0.1.4-cp314-cp314-win_amd64.whl |
Total release size: 584.8 kB
Release files / pyipcnode-0.1.4.tar.gz
| Download URL | pyipcnode-0.1.4.tar.gz |
|---|---|
| Size | 292.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
545510f29ba224d915d5b93160b59a0a50f4bee2d65ae92f557d27e167199ef5
|
|
BLAKE2b-256 checksum How to use checksums |
dc788f9160cb1f231e7fc3fd12f96190e2d3081d5948a2282d5d7851a45f62d2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.6
|
Release files / pyipcnode-0.1.4-cp314-cp314-win_amd64.whl
| Download URL | pyipcnode-0.1.4-cp314-cp314-win_amd64.whl |
|---|---|
| Size | 291.9 kB |
| Tags | CPython 3.14 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
f56cb71a279c628661c9160b32f8d10310c54be2c15a670a6b47526f9275a6b9
|
|
BLAKE2b-256 checksum How to use checksums |
a2a7d8970ba92d5a2d4b3d12069feaaaf0bd02457a65527396410e53a89f4757
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.6
|
Security for pyipcnode 0.1.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.