Skip to main content

mq-bridge Python bindings

Thin Python bindings for the Rust mq-bridge core.

Install

Pick exactly one distribution. Both install the same import path: mq_bridge.

Package Install Includes
Full pip install mq-bridge-py Basic set plus Kafka, AWS, gRPC, MongoDB, SQLx
Basic pip install mq-bridge-py-basic HTTP, NATS, MQTT, AMQP, WebSocket, ZeroMQ, middleware

Memory and file endpoints are always present in both packages. Use mq-bridge-py-basic when you want the lean all-platform wheel set. Use mq-bridge-py when you need Kafka or the heavier non-messaging backends.

The public API stays close to mq-bridge itself:

  • Route.from_yaml(path, name) loads one named route from a YAML file
  • Route.from_yaml_str(text, name) / Route.from_config(mapping, name) build a route from an in-memory YAML string or a Python dict, no file required
  • Route.with_handler(...) attaches a raw Message handler, with lazy json()/text() readers and with_json()/with_payload() response helpers
  • Route.add_handler(kind, ...) uses mq-bridge's kind dispatch and delivers decoded JSON
  • RetryableError and NonRetryableError let Python handlers signal retry intent
  • Publisher.from_yaml(path, name) (plus from_yaml_str / from_config) loads one named publisher
  • Publisher.send_json(...) and Publisher.request_json(...) serialize Python JSON values in Rust

The Python surface is synchronous and blocking. Tokio, broker I/O, routing, and batching all stay in Rust.

Config types and schema

mq-bridge-app can create and test route and endpoint JSON/YAML through its UI. It does not replace your Python code or handlers, but it is useful when you want a known-good connection and route shape before pasting the configuration into Python. Load the generated config with Route.from_config, Route.from_yaml, Publisher.from_config, or Publisher.from_yaml.

For the from_config / from_yaml_str mappings, mq_bridge.config ships TypedDict definitions so editors autocomplete the config keys (input, output, batch_size, every transport config, middleware, …):

from mq_bridge import Route
from mq_bridge.config import ConfigDocument

config: ConfigDocument = {
    "routes": {
        "orders": {
            "input": {"memory": {"topic": "orders.in", "capacity": 1600}},
            "output": {"response": {}},
            "batch_size": 128,
        }
    }
}
route = Route.from_config(config, "orders")

These types are generated from the JSON Schema, which the extension produces on demand from the Rust models — there is no checked-in schema copy to drift:

from mq_bridge import config_schema

schema = config_schema()        # the JSON Schema as a dict

config_schema() is handy for editor validation of YAML configs too — dump it to a file and point your # yaml-language-server: $schema= line at it. The types are regenerated with uv run python scripts/gen_config_types.py (a test fails if they drift from the schema).

Running a route

Route.run() blocks the calling thread until another thread calls stop() — it deploys the route and then parks. This is convenient for a process whose only job is the route, but it is a common trap: nothing after route.run() executes until the route stops.

To keep running Python code after the route is up, use start() (non-blocking) or the context-manager form:

route = Route.from_config(config, "orders_route").with_handler(handle)

# Non-blocking: deploys, returns, and runs on a background thread.
route.start()
publisher.send_json({"order_id": 42}, {"kind": "order.created"})
route.stop()
route.join()   # optional: wait for a clean shutdown

# Or scope it to a block — starts on enter, stops + joins on exit:
with Route.from_config(config, "orders_route").with_handler(handle):
    publisher.send_json({"order_id": 42}, {"kind": "order.created"})

Configuration/connection errors surface from start() itself, not from a background thread. run() remains available for the blocking single-route case.

Tuning (environment variables)

These knobs are read from the environment at startup:

Variable Default Effect
MQ_BRIDGE_PY_HANDLER_EXECUTOR worker worker runs handlers on a dedicated interpreter thread that coalesces queued batches under one GIL acquisition (best under load); direct calls the handler inline.
MQ_BRIDGE_PY_HANDLER_CONCURRENCY CPU count Max in-flight handler batches. 0 disables the limit.
MQ_BRIDGE_PY_GC_MODE default default leaves CPython's cyclic GC alone; count disables it and runs gc.collect() every N messages; off disables it entirely (pure refcounting).
MQ_BRIDGE_PY_GC_THRESHOLD 100000 Messages between collections when MQ_BRIDGE_PY_GC_MODE=count.

Local development

uv is a good fit here for the Python-side developer workflow, while maturin stays the build backend:

cd python/mq-bridge-py
uv sync --group dev --no-install-project
uv run maturin develop
uv run pytest -q

Performance smoke tests are skipped by default because they start routes and measure local throughput:

cd python/mq-bridge-py
MQ_BRIDGE_RUN_PERF_TESTS=1 uv run pytest -q -m performance

Examples

Raw message handler:

cd python/mq-bridge-py
uv run python examples/raw_route.py

Kind-based JSON handler:

cd python/mq-bridge-py
uv run python examples/json_route.py

Memory benchmark:

cd python/mq-bridge-py
uv run maturin develop --release
uv run python examples/bench_memory.py --messages 100000

Analysis

HTTP comparison benchmark, driven by a native load generator (wrk) so the client is never the bottleneck. It boots each server itself (mq-bridge in worker and direct executor modes, plus FastAPI, Starlette, Sanic, aiohttp, and FastStream when installed) and drives each with wrk:

cd python/mq-bridge-py
uv run maturin develop --release
uv sync --group bench   # optional Python HTTP peers
uv run python analysis/bench_http_native.py --connections 1,8,32 --duration 8

Requires wrk on PATH (brew install wrk). The FastStream target compares its ASGI custom-route path over Uvicorn; it is not a broker-backed subscriber/publisher benchmark. The examples use included sample configs or create temporary configs.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mq_bridge_py_basic-0.3.0.tar.gz (683.2 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mq_bridge_py_basic-0.3.0-cp38-abi3-win_amd64.whl (8.2 MB view details)

Uploaded CPython 3.8+Windows x86-64

mq_bridge_py_basic-0.3.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.0 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.17+ x86-64

mq_bridge_py_basic-0.3.0-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (7.8 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.17+ ARM64

mq_bridge_py_basic-0.3.0-cp38-abi3-macosx_11_0_arm64.whl (7.3 MB view details)

Uploaded CPython 3.8+macOS 11.0+ ARM64

mq_bridge_py_basic-0.3.0-cp38-abi3-macosx_10_12_x86_64.whl (7.7 MB view details)

Uploaded CPython 3.8+macOS 10.12+ x86-64

File details

Details for the file mq_bridge_py_basic-0.3.0.tar.gz.

File metadata

  • Download URL: mq_bridge_py_basic-0.3.0.tar.gz
  • Upload date:
  • Size: 683.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mq_bridge_py_basic-0.3.0.tar.gz
Algorithm Hash digest
SHA256 da05c6873535df291483d4da7eba2341a8e7485ef5214b64616ead3e17339595
MD5 49dbadbf810cd1a0eb50170ced4a9ac0
BLAKE2b-256 aefdd35c9dfea8b9589f572c5a150abd1afe45cd382f8d9e96714e8557f689ee

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.3.0.tar.gz:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.3.0-cp38-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.3.0-cp38-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 81666f81129bad975ada68c49ace7e70563043f4fa07d78cdfa310b60862e317
MD5 46be8ba9e4800bd9f25753b05bde618b
BLAKE2b-256 47a87bc173b3832f41d312554354dcb615cdf3db1448acabc087cea82e5b48d9

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.3.0-cp38-abi3-win_amd64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.3.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.3.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 64ec1d7d44de46eacd908758d773feaa1e4dfe1a98d10da73a9ac4897d055fe3
MD5 f8229e244d97e2e4b14861e9ed31ac78
BLAKE2b-256 c92646f86672a89fca9bc81795abb633c8a4db69227a3271f00fd06d7e4c3ae3

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.3.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.3.0-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.3.0-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 31ca17c3f543d4192d8eb29b79854205530a23c05cb6fb5d0e730df210500efd
MD5 7da2ab3cf6195e54a97b720981c54530
BLAKE2b-256 bea389559852a7ec134c43e7f12370f194a209425673221526274b5e1a5eee44

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.3.0-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.3.0-cp38-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.3.0-cp38-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 5611b137239bf52228d2d7b1e18b6f8502763f72d7f003501c1398c33bfd9c31
MD5 2d05e4a3d2505ba44e8b1bdef955854b
BLAKE2b-256 b0f106aa7cb4191d763200b04c4329d29f1fb9596ebb701cb3a94ef891c13232

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.3.0-cp38-abi3-macosx_11_0_arm64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mq_bridge_py_basic-0.3.0-cp38-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for mq_bridge_py_basic-0.3.0-cp38-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 02c78dccdedad769ab5c8da1fb3b0fb3994c57822cf135aa6b2ecd5024227489
MD5 ac2eeddda0901f874233dc20e7bbabc3
BLAKE2b-256 bac16fe7e6099c6ea5c3bd5e82a860dd99a4e749ec2a25b81d247a88f189e5d0

See more details on using hashes here.

Provenance

The following attestation bundles were made for mq_bridge_py_basic-0.3.0-cp38-abi3-macosx_10_12_x86_64.whl:

Publisher: publish-python.yml on marcomq/mq-bridge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page