humr-ipc 1.0.0

A Python library for shared memory IPC communication

humr-ipc

A Python library for shared-memory inter-process communication (IPC). Originally developed as the IPC layer for OpenHUMR — an open-source power grid simulation platform — it has no OpenHUMR-specific dependencies and works in any Python multi-processing application that needs fast, low-overhead communication between processes.

Features

• Shared memory transport — uses multiprocessing.shared_memory (zero-copy, no serialization overhead)
• Async-native — built on asyncio for non-blocking send/receive
• Pluggable middleware — swap the action handler without changing transport code
• Pluggable bundlers — control how payloads are framed before going onto the wire
• Cross-platform memory limits — auto-detects shared memory constraints on Windows, Linux, and WSL2
• Enum-based message types — with automatic JSON export
• Structured logging — colour-coded output via the built-in logger

Installation

# From PyPI:
pip install humr-ipc

# From GitHub:
pip install git+https://github.com/apteron-oss/humr-ipc.git

# For local development:
git clone https://github.com/apteron-oss/humr-ipc.git
pip install -e humr-ipc/[dev]

Quick start

Process A — creator (e.g. Python compute server)

import asyncio
from ipc import Communicator
from ipc.helpers.middleware.websocket import WebSocketMiddleware

async def handle_connection(websocket):
    middleware = WebSocketMiddleware(websocket)

    # Creates two named shared memory blocks; passes their names to Process B
    comm = Communicator(middleware, create_memory=True)

    print(comm.recv_memory_block.name)  # pass this to Process B as --recv
    print(comm.send_memory_block.name)  # pass this to Process B as --send

    await comm.listen_async()

Process B — consumer (e.g. spawned Rust/C++ binary or Python worker)

Process B attaches to the existing blocks by name:

comm = Communicator(
    middleware,
    create_memory=False,
    recv_memory_name="<name from Process A send block>",
    send_memory_name="<name from Process A recv block>",
    shared_memory_size=<size>,
)

In OpenHUMR the Rust binary reads block names from CLI arguments and uses the shared_memory crate to attach.

Architecture

Process A                           Process B
─────────────────────────────────────────────────
Communicator (creator)              Communicator (consumer)
  recv_memory_block ──────read───▶  send_memory_block
  send_memory_block ◀───write──────  recv_memory_block
  flag byte (index 0) coordinates read/write turns

Each block has a 1-byte flag at index 0. The writer sets the flag; the reader clears it after consuming. This avoids polling overhead while keeping the implementation dependency-free.

Middleware

Middleware handles what happens when a message arrives on the shared memory channel.

Class	Purpose
WebSocketMiddleware	Forward the payload to a WebSocket client in real time
ConsoleMiddleware	Print to stdout (debugging)
BlankMiddleware	No-op (testing / benchmarking)

All middleware classes implement ActionMiddleware and have a set_communicator() hook so they can send data back to the other process.

Bundlers

Bundlers frame outgoing data before writing it to shared memory.

Class	Purpose
WebSocketBundler	Wraps payload with action header for WebSocket routing
JSONBundler	Plain JSON framing

Message actions

from ipc.helpers.enums.ipc import IPCAction

IPCAction.BYPASS    # 0 — stream raw data directly, bypass logger
IPCAction.PRINT     # 1 — log message to console
IPCAction.TERMINATE # 2 — graceful shutdown signal

Module layout

ipc/
├── communicator.py          # Communicator class — main entry point
├── helpers/
│   ├── bundler/
│   │   ├── standard.py      # JSONBundler
│   │   └── websocket.py     # WebSocketBundler
│   ├── enums/
│   │   ├── base.py          # BaseEnum with JSON export
│   │   ├── ipc.py           # IPCAction
│   │   └── websocket.py     # WebSocket constants
│   ├── middleware/
│   │   ├── blank.py         # BlankMiddleware
│   │   ├── console.py       # ConsoleMiddleware
│   │   └── websocket.py     # WebSocketMiddleware
│   ├── random/
│   │   └── uuid.py          # Timestamped UUID generation
│   └── system/
│       └── shared_memory.py # Platform shared memory limit detection
└── logger/
    └── logger.py            # Structured colour-coded logger

Requirements

• Python 3.8+
• No external runtime dependencies (standard library only)

Origin

humr-ipc was extracted from OpenHUMR, where it bridges the Python Starlette compute server and the Rust thermal simulation binary using named shared memory blocks. It is maintained here as a standalone package so it can be versioned and reused independently.

License

See LICENSE.