ferryboat

In-process, Rust-core communication channel for Python. Zero TCP, zero IPC, zero subprocess overhead — just a lock-free channel living inside your Python process as a native extension.

• Speed: crossbeam-channel (sync) + tokio::sync::mpsc (async) primitives, GIL released on every blocking operation.
• No data loss: bounded channels with backpressure. Senders block or await; they never drop.
• Dual API: sync and async, same Rust core.
• Observability: structured Rust tracing events piped straight into Python logging.
• Ergonomic: type-annotated, context managers, iterators, async iterators.

Installation

pip install ferryboat

Usage

Sync API

import logging
import threading
import ferryboat

ferryboat.configure_logging(level="INFO", fmt="text")
logging.basicConfig(level=logging.DEBUG)

tx, rx = ferryboat.Channel.new(capacity=1024)

def producer() -> None:
    with tx:                                # closes sender on exit
        for i in range(10_000):
            tx.send(f"msg-{i}".encode())

threading.Thread(target=producer).start()

for msg in rx:                              # iterates until sender closes
    print(msg.decode())

Async API

import asyncio
import ferryboat

ferryboat.configure_logging(level="DEBUG")

async def producer(tx: ferryboat.AsyncSender) -> None:
    async with tx:
        for i in range(10_000):
            await tx.send(f"msg-{i}".encode())

async def consumer(rx: ferryboat.AsyncReceiver) -> None:
    async for msg in rx:                    # StopAsyncIteration on close
        print(msg.decode())

async def main() -> None:
    tx, rx = ferryboat.AsyncChannel.new(capacity=1024)
    await asyncio.gather(producer(tx), consumer(rx))

asyncio.run(main())

Batch receive (drain)

batch: list[bytes] = rx.drain()
process_batch(batch)

Timeout receive

try:
    msg = rx.recv_timeout(timeout_ms=500)
except TimeoutError:
    print("Nothing arrived in 500 ms")

Logging configuration

import ferryboat

# Text format (default) — readable in the terminal
ferryboat.configure_logging(level="DEBUG", fmt="text")

# JSON format — structured, for log aggregators
ferryboat.configure_logging(level="INFO", fmt="json")

# Or control the Rust log level via env var:
#   FERRYBOAT_LOG=warn python your_app.py

Environment variables:

Building from source

Prerequisites

• Rust ≥ 1.75 (install via rustup)
• Python ≥ 3.10
• maturin ≥ 1.4 (pip install maturin)

Development build

git clone https://github.com/your-org/ferryboat
cd ferryboat

python -m venv .venv
source .venv/bin/activate           # Windows: .venv\Scripts\activate

pip install maturin pytest pytest-asyncio
maturin develop --release

pytest tests/

Production wheel

maturin build --release
pip install target/wheels/ferryboat-*.whl

CI (GitHub Actions skeleton)

- uses: PyO3/maturin-action@v1
  with:
    command: build
    args: --release --out dist
- run: pip install dist/ferryboat-*.whl && pytest tests/

Design notes

• GIL discipline: sync blocking calls use py.allow_threads(); async calls use pyo3-async-runtimes::tokio::future_into_py so awaitables are native Python coroutines.
• Close semantics: dropping the sender (via close(), with tx:, or garbage collection) closes the channel. Receivers see ChannelClosedError when empty; iterators terminate cleanly.
• Observability: Rust tracing events are emitted as JSON on stderr, then parsed by a background reader thread and re-emitted on the ferryboat.rust Python logger. Users configure formatting on the ferryboat logger.

License

MIT

Benchmarks

Check here for latest benchmarks