Skip to main content

mq-bridge Python bindings

Thin Python bindings for the Rust mq-bridge core.

Install

Pick exactly one distribution. Both install the same import path: mq_bridge.

Package Install Includes
Full pip install mq-bridge-py Basic set plus Kafka, AWS, gRPC, MongoDB, SQLx
Basic pip install mq-bridge-py-basic HTTP, NATS, MQTT, AMQP, WebSocket, ZeroMQ, middleware

Memory and file endpoints are always present in both packages. Use mq-bridge-py-basic when you want the lean all-platform wheel set. Use mq-bridge-py when you need Kafka or the heavier non-messaging backends.

The public API stays close to mq-bridge itself:

  • Route.from_yaml(path, name) loads one named route from a YAML file
  • Route.from_yaml_str(text, name) / Route.from_config(mapping, name) build a route from an in-memory YAML string or a Python dict, no file required
  • Route.with_handler(...) attaches a raw Message handler, with lazy json()/text() readers and with_json()/with_payload() response helpers
  • Route.add_handler(kind, ...) uses mq-bridge's kind dispatch and delivers decoded JSON
  • RetryableError and NonRetryableError let Python handlers signal retry intent
  • Publisher.from_yaml(path, name) (plus from_yaml_str / from_config) loads one named publisher
  • Publisher.send_json(...) and Publisher.request_json(...) serialize Python JSON values in Rust

The Python surface is synchronous and blocking. Tokio, broker I/O, routing, and batching all stay in Rust.

Config types and schema

mq-bridge-app can create and test route and endpoint JSON/YAML through its UI. It does not replace your Python code or handlers, but it is useful when you want a known-good connection and route shape before pasting the configuration into Python. Load the generated config with Route.from_config, Route.from_yaml, Publisher.from_config, or Publisher.from_yaml.

For the from_config / from_yaml_str mappings, mq_bridge.config ships TypedDict definitions so editors autocomplete the config keys (input, output, batch_size, every transport config, middleware, …):

from mq_bridge import Route
from mq_bridge.config import ConfigDocument

config: ConfigDocument = {
    "routes": {
        "orders": {
            "input": {"memory": {"topic": "orders.in", "capacity": 1600}},
            "output": {"response": {}},
            "batch_size": 128,
        }
    }
}
route = Route.from_config(config, "orders")

These types are generated from the JSON Schema, which the extension produces on demand from the Rust models — there is no checked-in schema copy to drift:

from mq_bridge import config_schema

schema = config_schema()        # the JSON Schema as a dict

config_schema() is handy for editor validation of YAML configs too — dump it to a file and point your # yaml-language-server: $schema= line at it. The types are regenerated with uv run python scripts/gen_config_types.py (a test fails if they drift from the schema).

Running a route

Route.run() blocks the calling thread until another thread calls stop() — it deploys the route and then parks. This is convenient for a process whose only job is the route, but it is a common trap: nothing after route.run() executes until the route stops.

To keep running Python code after the route is up, use start() (non-blocking) or the context-manager form:

route = Route.from_config(config, "orders_route").with_handler(handle)

# Non-blocking: deploys, returns, and runs on a background thread.
route.start()
publisher.send_json({"order_id": 42}, {"kind": "order.created"})
route.stop()
route.join()   # optional: wait for a clean shutdown

# Or scope it to a block — starts on enter, stops + joins on exit:
with Route.from_config(config, "orders_route").with_handler(handle):
    publisher.send_json({"order_id": 42}, {"kind": "order.created"})

Configuration/connection errors surface from start() itself, not from a background thread. run() remains available for the blocking single-route case.

Tuning (environment variables)

These knobs are read from the environment at startup:

Variable Default Effect
MQ_BRIDGE_PY_HANDLER_EXECUTOR worker worker runs handlers on a dedicated interpreter thread that coalesces queued batches under one GIL acquisition (best under load); direct calls the handler inline.
MQ_BRIDGE_PY_HANDLER_CONCURRENCY CPU count Max in-flight handler batches. 0 disables the limit.
MQ_BRIDGE_PY_GC_MODE default default leaves CPython's cyclic GC alone; count disables it and runs gc.collect() every N messages; off disables it entirely (pure refcounting).
MQ_BRIDGE_PY_GC_THRESHOLD 100000 Messages between collections when MQ_BRIDGE_PY_GC_MODE=count.

Local development

uv is a good fit here for the Python-side developer workflow, while maturin stays the build backend:

cd python/mq-bridge-py
uv sync --group dev --no-install-project
uv run maturin develop
uv run pytest -q

Performance smoke tests are skipped by default because they start routes and measure local throughput:

cd python/mq-bridge-py
MQ_BRIDGE_RUN_PERF_TESTS=1 uv run pytest -q -m performance

Examples

Raw message handler:

cd python/mq-bridge-py
uv run python examples/raw_route.py

Kind-based JSON handler:

cd python/mq-bridge-py
uv run python examples/json_route.py

Memory benchmark:

cd python/mq-bridge-py
uv run maturin develop --release
uv run python examples/bench_memory.py --messages 100000

Analysis

HTTP comparison benchmark, driven by a native load generator (wrk) so the client is never the bottleneck. It boots each server itself (mq-bridge in worker and direct executor modes, plus FastAPI, Starlette, Sanic, aiohttp, and FastStream when installed) and drives each with wrk:

cd python/mq-bridge-py
uv run maturin develop --release
uv sync --group bench   # optional Python HTTP peers
uv run python analysis/bench_http_native.py --connections 1,8,32 --duration 8

Requires wrk on PATH (brew install wrk). The FastStream target compares its ASGI custom-route path over Uvicorn; it is not a broker-backed subscriber/publisher benchmark. The examples use included sample configs or create temporary configs.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mq_bridge_py_basic-0.2.21.tar.gz (664.4 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mq_bridge_py_basic-0.2.21-cp38-abi3-win_amd64.whl (8.1 MB view details)

Uploaded CPython 3.8+Windows x86-64

mq_bridge_py_basic-0.2.21-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.9 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.17+ x86-64

mq_bridge_py_basic-0.2.21-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (7.7 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.17+ ARM64

mq_bridge_py_basic-0.2.21-cp38-abi3-macosx_11_0_arm64.whl (7.3 MB view details)

Uploaded CPython 3.8+macOS 11.0+ ARM64

mq_bridge_py_basic-0.2.21-cp38-abi3-macosx_10_12_x86_64.whl (7.6 MB view details)

Uploaded CPython 3.8+macOS 10.12+ x86-64

File details

Details for the file mq_bridge_py_basic-0.2.21.tar.gz.

File metadata

  • Download URL: mq_bridge_py_basic-0.2.21.tar.gz
  • Upload date:
  • Size: 664.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mq_bridge_py_basic-0.2.21.tar.gz
Algorithm Hash digest
SHA256 fd2228bfd431af1d59c81fbdaefc1617d07f0c9c852d50142b84ecbef8456138
MD5 858252c2cb87348492859cabc14d9105
BLAKE2b-256 36dd5cc1fddba660e81ac7785e68a4560fdf465f55a3e454603264d781ce58f7

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.2.21.tar.gz:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.2.21-cp38-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.2.21-cp38-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 633fcdb5695dc05d1cbaa8ad93e506df3235e3528660728a68fd9948ffc2573b
MD5 882897287399f6535cbdf64e1a845e5f
BLAKE2b-256 e68f913654e6d28576d274cadb2e006dc0d432d08a97a8225a0cd3f3057ea57d

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.2.21-cp38-abi3-win_amd64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.2.21-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.2.21-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d2d2360d563bf28851ae4e1577a3bf639bd8025a6dc4deb493fe38dc84a57e7e
MD5 f0fd568679593e898fb96b0111b5de0a
BLAKE2b-256 25b33e3fdb8649c6ee2f6286d05e14c6312b8698d58de05ebba9dd082ac43b08

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.2.21-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.2.21-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.2.21-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 713442540b4fb3de5468c99868a9303608eef222a58cbc87c26ab1d602436c61
MD5 3be035e622df0dc1ba4ee797154f4b28
BLAKE2b-256 b677e81c191e6e7854f7fd362f15860e2d1dca72bc09262ee13cf373c685a3d6

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.2.21-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.2.21-cp38-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.2.21-cp38-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 ab8299c391fd4cf47e1f8d3591a43bfac29a893e625c705046cea300bf77785a
MD5 771c4b976da6055d1cfee2857564d7b6
BLAKE2b-256 1544631fb8dcfe74599ac3110a32a3f7c67c484a8a66fedc7aa255aa3ffc6d02

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.2.21-cp38-abi3-macosx_11_0_arm64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.2.21-cp38-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.2.21-cp38-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 32f98c55d198fe55e0d26bf4ff4a29b9cf52a8bef0f2596f709615b334681365
MD5 db4434c0fd26257da9383dd22768ba21
BLAKE2b-256 6630fe20d883afbcd4ede1499b0fe968a9715d4b2af6eb8674d2c69990048f63

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.2.21-cp38-abi3-macosx_10_12_x86_64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page