Skip to main content

pyipcnode

Peer-to-peer method calls between Python processes on the same machine. Processes join a named system, register methods, look up peers, and call them over shared memory. There is no server process.

Calls go through a shared directory plus per-node shared-memory mailboxes, not sockets or pipes. That keeps same-machine RPC cheap:

  • No broker. Any node can call any other once it knows (node_name, pid). Nothing sits in the middle, and nothing has to stay up as a server.
  • Low latency. A call is a write into the peer's mailbox and a wake-up. There is no TCP/loopback stack, no listen port, and no extra process hop.
  • Local discovery. Peers in the same system_name show up in a shared table. list_nodes() is a local read; you do not poll a name service.
  • Compact payloads. Arguments travel as little-endian binary, not JSON text over a socket. Large arrays stay packed (u32 count + tightly packed elements).
  • Isolation by name. Different system_name values are separate systems on the same machine. Processes that should not talk simply do not share a name.

This is for same-OS, same-machine processes. It is not a network RPC library.

Install

pip install pyipcnode

Requires Python 3.14. Nodes must run on the same OS and the same machine, and they must use the same system_name.

Quick start

Two processes, same system_name. One registers add; the other finds it and calls it.

Callee

from pyipcnode import IpcNode, i4

def add(a: i4, b: i4) -> i4:
    return a + b

with IpcNode("worker", "mysystem", "box_worker", 64 * 1024) as node:
    node.register("add", add)
    input("running, press Enter to exit\n")

Caller

from pyipcnode import (
    STATUS_OK,
    IpcNode,
    decode_result,
    encode_args,
    find_method,
    parse_list_methods,
)

with IpcNode("client", "mysystem", "box_client", 64 * 1024) as node:
    peer = next(n for n in node.list_nodes() if n.node_name == "worker")
    listed = node.call_raw(peer.node_name, peer.pid, "sys.list_methods", b"", 1000).result()
    method = find_method(parse_list_methods(listed.result), "add")
    result = node.call_raw(
        peer.node_name, peer.pid, "add", encode_args(method, 2, 3), 1000
    ).result()
    if result.status == STATUS_OK:
        print(decode_result(method["return"], result.result))  # 5

Create a node

IpcNode(node_name, system_name, recv_shm_name, recv_shm_bytes)
Argument Meaning
node_name Logical name of this node. Duplicates are allowed; identity is (node_name, pid).
system_name Communication system. Required, non-empty, [A-Za-z0-9_] only. Nodes with different names cannot see each other.
recv_shm_name Base name of this node's receive mailbox. The library appends _ and the process pid.
recv_shm_bytes Mailbox size in bytes. Required; no default.

node.pid is read-only and taken from the current process. Construction failures (bad names, mailbox too small, system table full, and similar) raise RuntimeError.

Always close the node. A live node holds a slot in the system table:

with IpcNode("worker", "mysystem", "box_worker", 64 * 1024) as node:
    ...

node.close()  # safe to call more than once

After close(), register, call_raw, and list_nodes raise RuntimeError.

Register methods

Prefer type annotations. Every parameter and the return type must be annotated with a marker from this package. Bare int, str, or float is rejected.

from pyipcnode import IpcNode, char_p, i4

def add(a: i4, b: i4) -> i4:
    return a + b

def echo(text: char_p) -> char_p:
    return text

def ping() -> None:
    return None

def sum_vec(xs: list[i4]) -> i4:
    return sum(xs)

node.register("add", add)   # recommended: explicit name
node.register(ping)         # uses ping.__name__

register returns True on success. Name clashes, empty names, and names starting with sys. return False. Missing or unsupported annotations raise TypeError and nothing is registered.

Built-in methods sys.list_methods and sys.list_nodes are registered at startup and cannot be overridden.

Annotation markers

Annotation Wire type Python value
i1 i2 i4 i8 signed integers int (range-checked)
u1 u2 u4 u8 unsigned integers int (range-checked)
f4 f8 IEEE-754 float / double float
char_p UTF-8 string str
list[i4] (and other integer/float markers) variable-length array list

list elements must be integer or float markers. Nested lists and list[char_p] are not supported. bool is not a wire type; True is not accepted as an integer. A typed method returns nothing (-> None) or exactly one value.

Call a peer

call_raw is the only send API. It returns a concurrent.futures.Future immediately. The IPC timeout is timeout_ms (required, milliseconds, >= 0). Failures are reported on CallResult.status; they do not raise.

future = node.call_raw(node_name, pid, method, args, timeout_ms)
cr = future.result()          # wait until the IPC finishes
cr = future.result(timeout=2) # extra Python-side wait; does not cancel the call

Future.cancel() does not cancel an in-flight call and returns False.

Discover peers and methods

peers = node.list_nodes()  # local snapshot: list[NodeInfo(node_name, pid)]

list_nodes() reads the local system table. It does not probe whether a peer is still alive.

To inspect a peer's methods:

cr = node.call_raw(peer.node_name, peer.pid, "sys.list_methods", b"", 1000).result()
methods = parse_list_methods(cr.result)
spec = find_method(methods, "add")

find_method returns None when the name is missing.

Pack arguments and unpack results

Use the peer's method description (from sys.list_methods, or a dict you already have):

args = encode_args(spec, 2, 3)
cr = node.call_raw(peer.node_name, peer.pid, "add", args, 1000).result()
if cr.status == STATUS_OK:
    value = decode_result(spec["return"], cr.result)

encode_args accepts either the full method dict or its params list. Values must match the contract in count and order.

Contract Accepted values
integers int (not bool), in range
f4 / f8 int or float
char* str
short T* list of scalars
large T* array.array with a matching typecode, or bytes / bytearray / memoryview

decode_result handles empty, scalar, and string returns. Array returns (T*) must use decode_array: u1* becomes bytes, other element types become array.array.

import array

args = encode_args(spec, array.array("i", [1, 2, 3]))
cr = node.call_raw(peer.node_name, peer.pid, "sum_vec", args, 1000).result()

Status codes

Future.result() always returns a CallResult unless the extension itself fails. Check status; do not treat a non-zero status as an exception.

Constant Value Meaning
STATUS_OK 0 Success. result holds the return payload.
STATUS_TIMEOUT 1 No reply before timeout_ms.
STATUS_EXEC_FAILED 2 Method missing, handler raised, peer gone, or unpack failed on the callee.
STATUS_DELIVER_FAILED 3 Peer's mailbox is full. Retry yourself; the library does not.

On failure, CallResult.result is b"".

Raw bytes API

Custom layouts, multiple return values, or types outside the marker table stay on the bytes path. call_raw still takes and returns raw bytes.

import struct

def add_fn(args: bytes) -> bytes:
    a, b = struct.unpack_from("<ii", args, 0)
    return struct.pack("<i", a + b)

node.register(
    {
        "type": "method",
        "name": "add",
        "params": [{"type": "i4"}, {"type": "i4"}],
        "return": [{"type": "i4"}],
    },
    add_fn,
)

cr = node.call_raw(peer.node_name, peer.pid, "add", struct.pack("<ii", 1, 2), 1000).result()

Typed register and raw register(dict, fn) can be mixed on the same node.

Threading

Registered functions run on a background thread (the library acquires the GIL). Do not call Future.result() on a same-node in-flight call from inside a registered function; that can deadlock. Waiting from another thread is fine.

Two nodes must be able to call each other at the same time: while one side waits on result(), the other side's registered functions must still run.

Notes

  • Integers and floats on the wire are little-endian.
  • char* is UTF-8 with a trailing NUL; the length prefix includes that byte.
  • T* is a little-endian u32 element count, then packed elements.
  • The library does not auto-fetch or cache a peer's method list.
  • __del__ calls close() as a fallback. Do not rely on garbage collection to release the node.

Metadata

Release files for pyipcnode 0.1.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pyipcnode 0.1.2
File Size Uploaded Provenance Actions
pyipcnode-0.1.2.tar.gz 2.7 MB Without provenance Details for pyipcnode-0.1.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for pyipcnode 0.1.2
File Interpreter ABI Platform Actions
pyipcnode-0.1.2-cp314-cp314-win_amd64.whl CPython 3.14 CPython 3.14 Windows x86-64 Details for pyipcnode-0.1.2-cp314-cp314-win_amd64.whl

Total release size: 5.5 MB

Release files / pyipcnode-0.1.2.tar.gz

Download URL pyipcnode-0.1.2.tar.gz
Size 2.7 MB
Tags Source
SHA-256 checksum
How to use checksums
44547095c42204cca07e03e2414a6cc57640b83b0f7879095989e8c2bd085fa1
BLAKE2b-256 checksum
How to use checksums
62e4f661fcbb3914e93cc9494351d94c158957009eac3c0b672f40a473267204
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.6

Release files / pyipcnode-0.1.2-cp314-cp314-win_amd64.whl

Download URL pyipcnode-0.1.2-cp314-cp314-win_amd64.whl
Size 2.8 MB
Tags CPython 3.14 Windows x86-64
SHA-256 checksum
How to use checksums
72456906301ae4500e3fedc8b56aa0abeca4df8edd0fb6ee1d1d9abc69d980df
BLAKE2b-256 checksum
How to use checksums
83818818a1b6120cf7ffb28a96624c3af5aa91cc4937b9439e451cd45b43ea56
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.6

Security for pyipcnode 0.1.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page