Skip to main content

mq-bridge Python bindings

Thin Python bindings for the Rust mq-bridge core.

Install

Pick exactly one distribution. Both install the same import path: mq_bridge.

Package Install Includes
Full pip install mq-bridge-py Basic set plus Kafka, AWS, gRPC, MongoDB, SQLx
Basic pip install mq-bridge-py-basic HTTP, NATS, MQTT, AMQP, WebSocket, ZeroMQ, middleware

Memory and file endpoints are always present in both packages. Use mq-bridge-py-basic when you want the lean all-platform wheel set. Use mq-bridge-py when you need Kafka or the heavier non-messaging backends.

The public API stays close to mq-bridge itself:

  • Route.from_yaml(path, name) loads one named route from a YAML file
  • Route.from_yaml_str(text, name) / Route.from_config(mapping, name) build a route from an in-memory YAML string or a Python dict, no file required
  • Route.with_handler(...) attaches a raw Message handler, with lazy json()/text() readers and with_json()/with_payload() response helpers
  • Route.add_handler(kind, ...) uses mq-bridge's kind dispatch and delivers decoded JSON
  • RetryableError and NonRetryableError let Python handlers signal retry intent
  • Publisher.from_yaml(path, name) (plus from_yaml_str / from_config) loads one named publisher
  • Publisher.send_json(...) and Publisher.request_json(...) serialize Python JSON values in Rust

The Python surface is synchronous and blocking. Tokio, broker I/O, routing, and batching all stay in Rust.

Config types and schema

mq-bridge-app can create and test route and endpoint JSON/YAML through its UI. It does not replace your Python code or handlers, but it is useful when you want a known-good connection and route shape before pasting the configuration into Python. Load the generated config with Route.from_config, Route.from_yaml, Publisher.from_config, or Publisher.from_yaml.

For the from_config / from_yaml_str mappings, mq_bridge.config ships TypedDict definitions so editors autocomplete the config keys (input, output, batch_size, every transport config, middleware, …):

from mq_bridge import Route
from mq_bridge.config import ConfigDocument

config: ConfigDocument = {
    "routes": {
        "orders": {
            "input": {"memory": {"topic": "orders.in", "capacity": 1600}},
            "output": {"response": {}},
            "batch_size": 128,
        }
    }
}
route = Route.from_config(config, "orders")

These types are generated from the JSON Schema, which the extension produces on demand from the Rust models — there is no checked-in schema copy to drift:

from mq_bridge import config_schema

schema = config_schema()        # the JSON Schema as a dict

config_schema() is handy for editor validation of YAML configs too — dump it to a file and point your # yaml-language-server: $schema= line at it. The types are regenerated with uv run python scripts/gen_config_types.py (a test fails if they drift from the schema).

Running a route

Route.run() blocks the calling thread until another thread calls stop() — it deploys the route and then parks. This is convenient for a process whose only job is the route, but it is a common trap: nothing after route.run() executes until the route stops.

To keep running Python code after the route is up, use start() (non-blocking) or the context-manager form:

route = Route.from_config(config, "orders_route").with_handler(handle)

# Non-blocking: deploys, returns, and runs on a background thread.
route.start()
publisher.send_json({"order_id": 42}, {"kind": "order.created"})
route.stop()
route.join()   # optional: wait for a clean shutdown

# Or scope it to a block — starts on enter, stops + joins on exit:
with Route.from_config(config, "orders_route").with_handler(handle):
    publisher.send_json({"order_id": 42}, {"kind": "order.created"})

Configuration/connection errors surface from start() itself, not from a background thread. run() remains available for the blocking single-route case.

Tuning (environment variables)

These knobs are read from the environment at startup:

Variable Default Effect
MQ_BRIDGE_PY_HANDLER_EXECUTOR worker worker runs handlers on a dedicated interpreter thread that coalesces queued batches under one GIL acquisition (best under load); direct calls the handler inline.
MQ_BRIDGE_PY_HANDLER_CONCURRENCY CPU count Max in-flight handler batches. 0 disables the limit.
MQ_BRIDGE_PY_GC_MODE default default leaves CPython's cyclic GC alone; count disables it and runs gc.collect() every N messages; off disables it entirely (pure refcounting).
MQ_BRIDGE_PY_GC_THRESHOLD 100000 Messages between collections when MQ_BRIDGE_PY_GC_MODE=count.

Local development

uv is a good fit here for the Python-side developer workflow, while maturin stays the build backend:

cd python/mq-bridge-py
uv sync --group dev --no-install-project
uv run maturin develop
uv run pytest -q

Performance smoke tests are skipped by default because they start routes and measure local throughput:

cd python/mq-bridge-py
MQ_BRIDGE_RUN_PERF_TESTS=1 uv run pytest -q -m performance

Examples

Raw message handler:

cd python/mq-bridge-py
uv run python examples/raw_route.py

Kind-based JSON handler:

cd python/mq-bridge-py
uv run python examples/json_route.py

Memory benchmark:

cd python/mq-bridge-py
uv run maturin develop --release
uv run python examples/bench_memory.py --messages 100000

Analysis

HTTP comparison benchmark, driven by a native load generator (wrk) so the client is never the bottleneck. It boots each server itself (mq-bridge in worker and direct executor modes, plus FastAPI, Starlette, Sanic, aiohttp, and FastStream when installed) and drives each with wrk:

cd python/mq-bridge-py
uv run maturin develop --release
uv sync --group bench   # optional Python HTTP peers
uv run python analysis/bench_http_native.py --connections 1,8,32 --duration 8

Requires wrk on PATH (brew install wrk). The FastStream target compares its ASGI custom-route path over Uvicorn; it is not a broker-backed subscriber/publisher benchmark. The examples use included sample configs or create temporary configs.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mq_bridge_py_basic-0.2.20.tar.gz (656.3 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mq_bridge_py_basic-0.2.20-cp38-abi3-win_amd64.whl (8.1 MB view details)

Uploaded CPython 3.8+Windows x86-64

mq_bridge_py_basic-0.2.20-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.9 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.17+ x86-64

mq_bridge_py_basic-0.2.20-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (7.7 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.17+ ARM64

mq_bridge_py_basic-0.2.20-cp38-abi3-macosx_11_0_arm64.whl (7.3 MB view details)

Uploaded CPython 3.8+macOS 11.0+ ARM64

mq_bridge_py_basic-0.2.20-cp38-abi3-macosx_10_12_x86_64.whl (7.6 MB view details)

Uploaded CPython 3.8+macOS 10.12+ x86-64

File details

Details for the file mq_bridge_py_basic-0.2.20.tar.gz.

File metadata

  • Download URL: mq_bridge_py_basic-0.2.20.tar.gz
  • Upload date:
  • Size: 656.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mq_bridge_py_basic-0.2.20.tar.gz
Algorithm Hash digest
SHA256 4017e6883bd73166b7c7eea0da7198ec08e7832e2d391e0692ab5a51d487f928
MD5 07c0d1c5914f92f0f08a30b53f799ec3
BLAKE2b-256 428e45a9b3166f31688a2fb4d1d1eadf648c82fa77f11cd1819788565473f298

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.2.20.tar.gz:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.2.20-cp38-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.2.20-cp38-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 e45b1ba52202f1ce468df84f9292f9c5133d81c3c3d6f3f9fc71d2b901a3d64b
MD5 3e7d36add10a8a242dc0d25bed4ba504
BLAKE2b-256 5a9c27b9983629ab5f2fcb550d7af57f07d4a96a39469f023124894a484991da

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.2.20-cp38-abi3-win_amd64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.2.20-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.2.20-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f38118eba53caaeb898728f2d8bd0c0903e1e08ff96085e663677e5a2390e591
MD5 89901f5f84c8618bffaaea14393936a9
BLAKE2b-256 86113b420bca3cc80a4eed9a96e1f4270dc20d88bd7506b80798b51aa0b27fe5

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.2.20-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.2.20-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.2.20-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 13f5c4e8621deb0c7c808f665f5bfba383daa1b33fb363684ae784afe068313e
MD5 21144832a6c3792f71ad4fa7ce81d871
BLAKE2b-256 7da0e33a6f71e44ed3bdb640a17b544084d4d5f0ec6f02348f875d3fc5f48f13

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.2.20-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.2.20-cp38-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.2.20-cp38-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f809e72bba3d72bb7b4089b5a2e883a8a0680658cdcc5a176ee4843e3f05aeed
MD5 150ef7a24f71ee5f8bf45f22d9375dfb
BLAKE2b-256 2db77d4c908438ed07d0df84738d1ccea3d90c53b4b4e42ddf02b2095915d8ae

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.2.20-cp38-abi3-macosx_11_0_arm64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.2.20-cp38-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.2.20-cp38-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 a5268676c9179f2c5a9e7ebad2b28c45a6ba389eb6f6fe9d48790b5dc52c5db8
MD5 fc15da6e31d159f059370a85f1d36dfa
BLAKE2b-256 3655fbab040e0ec3afd920d151643ad677f85d16d22495949e40c0ac49d88dc9

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.2.20-cp38-abi3-macosx_10_12_x86_64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page