MSGQ

High-performance pub/sub messaging, made simple.
For Python, C, and C++.

Quickstart · Examples · Discord

MSGQ lets programs on the same machine exchange messages. A publisher sends messages to a named endpoint, and subscribers listen on that same endpoint. Each endpoint supports one publisher and multiple subscribers.

MSGQ is a generic high performance IPC pub sub system with a single publisher and multiple subscribers. It uses a ring buffer in shared memory to efficiently read and write data. Each read requires a copy. Writing can be done without a copy, as long as the size of the data is known in advance. This library also provides a spoofed implementation that can be used for deterministic testing, and visionipc, an IPC system specifically for large contiguous buffers (like images/video).

_{1 KiB cross-process ping-pong on x86 Linux. Benchmark script.}

Quickstart

python -m pip install git+https://github.com/commaai/msgq.git

From a local checkout, run the publisher and subscriber in separate terminals:

python examples/publisher.py   # terminal 1
python examples/subscriber.py  # terminal 2

The subscriber prints Hello from MSGQ! once per second.

The core API sends and receives bytes:

import msgq

publisher = msgq.pub_sock("hello")
subscriber = msgq.sub_sock("hello")
publisher.send(b"Hello from MSGQ!")
print(subscriber.receive())  # b'Hello from MSGQ!'

Contributing

Issues and pull requests are welcome on GitHub. Run ./test.sh to build, lint, and test the package.

License

MSGQ is available under the MIT License.

Under the hood

Storage

The storage for the queue consists of an area of metadata, and the actual buffer. The metadata contains:

A counter to the number of readers that are active
A pointer to the head of the queue for writing. From now on referred to as write pointer
A cycle counter for the writer. This counter is incremented when the writer wraps around
N pointers, pointing to the current read position for all the readers. From now on referred to as read pointer
N counters, counting the number of cycles for all the readers
N booleans, indicating validity for all the readers. From now on referred to as validity flag

The counter and the pointer are both 32 bit values, packed into 64 bit so they can be read and written atomically.

The data buffer is a ring buffer. All messages are prefixed by an 8 byte size field, followed by the data. A size of -1 indicates a wrap-around, and means the next message is stored at the beginning of the buffer.

Writing

Writing involves the following steps:

Check if the area that is to be written overlaps with any of the read pointers, mark those readers as invalid by clearing the validity flag.
Write the message
Increase the write pointer by the size of the message

In case there is not enough space at the end of the buffer, a special empty message with a prefix of -1 is written. The cycle counter is incremented by one. In this case step 1 will check there are no read pointers pointing to the remainder of the buffer. Then another write cycle will start with the actual message.

There always needs to be 8 bytes of empty space at the end of the buffer. By doing this there is always space to write the -1.

Reset reader

When the reader is lagging too much behind the read pointer becomes invalid and no longer points to the beginning of a valid message. To reset a reader to the current write pointer, the following steps are performed:

Set valid flag
Set read cycle counter to that of the writer
Set read pointer to write pointer

Reading

Reading involves the following steps:

Read the size field at the current read pointer
Read the validity flag
Copy the data out of the buffer
Increase the read pointer by the size of the message
Check the validity flag again

Before starting the copy, the valid flag is checked. This is to prevent a race condition where the size prefix was invalid, and the read could read outside of the buffer. Make sure that step 1 and 2 are not reordered by your compiler or CPU.

If a writer overwrites the data while it's being copied out, the data will be invalid. Therefore the validity flag is also checked after reading it. The order of step 4 and 5 does not matter.

If at steps 2 or 5 the validity flag is not set, the reader is reset. Any data that was already read is discarded. After the reader is reset, the reading starts from the beginning.

If a message with size -1 is encountered, step 3 and 4 are replaced by increasing the cycle counter and setting the read pointer to the beginning of the buffer. After that another read is performed.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

msgq_ipc-1.0.tar.gz (35.5 kB view details)

Uploaded Sep 6, 2026 Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

msgq_ipc-1.0-cp314-cp314-musllinux_1_2_x86_64.whl (2.7 MB view details)

Uploaded Sep 6, 2026 CPython 3.14musllinux: musl 1.2+ x86-64

msgq_ipc-1.0-cp314-cp314-musllinux_1_2_aarch64.whl (2.6 MB view details)

Uploaded Sep 6, 2026 CPython 3.14musllinux: musl 1.2+ ARM64

msgq_ipc-1.0-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (1.7 MB view details)

Uploaded Sep 6, 2026 CPython 3.14manylinux: glibc 2.24+ x86-64manylinux: glibc 2.28+ x86-64

msgq_ipc-1.0-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl (1.7 MB view details)

Uploaded Sep 6, 2026 CPython 3.14manylinux: glibc 2.24+ ARM64manylinux: glibc 2.28+ ARM64

msgq_ipc-1.0-cp314-cp314-macosx_11_0_arm64.whl (240.2 kB view details)

Uploaded Sep 6, 2026 CPython 3.14macOS 11.0+ ARM64

msgq_ipc-1.0-cp314-cp314-macosx_10_15_x86_64.whl (245.8 kB view details)

Uploaded Sep 6, 2026 CPython 3.14macOS 10.15+ x86-64

msgq_ipc-1.0-cp313-cp313-musllinux_1_2_x86_64.whl (2.7 MB view details)

Uploaded Sep 6, 2026 CPython 3.13musllinux: musl 1.2+ x86-64

msgq_ipc-1.0-cp313-cp313-musllinux_1_2_aarch64.whl (2.6 MB view details)

Uploaded Sep 6, 2026 CPython 3.13musllinux: musl 1.2+ ARM64

msgq_ipc-1.0-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (1.7 MB view details)

Uploaded Sep 6, 2026 CPython 3.13manylinux: glibc 2.24+ x86-64manylinux: glibc 2.28+ x86-64

msgq_ipc-1.0-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl (1.7 MB view details)

Uploaded Sep 6, 2026 CPython 3.13manylinux: glibc 2.24+ ARM64manylinux: glibc 2.28+ ARM64

msgq_ipc-1.0-cp313-cp313-macosx_11_0_arm64.whl (239.1 kB view details)

Uploaded Sep 6, 2026 CPython 3.13macOS 11.0+ ARM64

msgq_ipc-1.0-cp313-cp313-macosx_10_13_x86_64.whl (245.1 kB view details)

Uploaded Sep 6, 2026 CPython 3.13macOS 10.13+ x86-64

msgq_ipc-1.0-cp312-cp312-musllinux_1_2_x86_64.whl (2.7 MB view details)

Uploaded Sep 6, 2026 CPython 3.12musllinux: musl 1.2+ x86-64

msgq_ipc-1.0-cp312-cp312-musllinux_1_2_aarch64.whl (2.6 MB view details)

Uploaded Sep 6, 2026 CPython 3.12musllinux: musl 1.2+ ARM64

msgq_ipc-1.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (1.8 MB view details)

Uploaded Sep 6, 2026 CPython 3.12manylinux: glibc 2.24+ x86-64manylinux: glibc 2.28+ x86-64

msgq_ipc-1.0-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl (1.7 MB view details)

Uploaded Sep 6, 2026 CPython 3.12manylinux: glibc 2.24+ ARM64manylinux: glibc 2.28+ ARM64

msgq_ipc-1.0-cp312-cp312-macosx_11_0_arm64.whl (237.1 kB view details)

Uploaded Sep 6, 2026 CPython 3.12macOS 11.0+ ARM64

msgq_ipc-1.0-cp312-cp312-macosx_10_13_x86_64.whl (242.9 kB view details)

Uploaded Sep 6, 2026 CPython 3.12macOS 10.13+ x86-64

msgq_ipc-1.0-cp311-cp311-musllinux_1_2_x86_64.whl (2.7 MB view details)

Uploaded Sep 6, 2026 CPython 3.11musllinux: musl 1.2+ x86-64

msgq_ipc-1.0-cp311-cp311-musllinux_1_2_aarch64.whl (2.6 MB view details)

Uploaded Sep 6, 2026 CPython 3.11musllinux: musl 1.2+ ARM64

msgq_ipc-1.0-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (1.7 MB view details)

Uploaded Sep 6, 2026 CPython 3.11manylinux: glibc 2.24+ x86-64manylinux: glibc 2.28+ x86-64

msgq_ipc-1.0-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl (1.7 MB view details)

Uploaded Sep 6, 2026 CPython 3.11manylinux: glibc 2.24+ ARM64manylinux: glibc 2.28+ ARM64

msgq_ipc-1.0-cp311-cp311-macosx_11_0_arm64.whl (228.4 kB view details)

Uploaded Sep 6, 2026 CPython 3.11macOS 11.0+ ARM64

msgq_ipc-1.0-cp311-cp311-macosx_10_9_x86_64.whl (231.6 kB view details)

Uploaded Sep 6, 2026 CPython 3.11macOS 10.9+ x86-64

File details

Details for the file msgq_ipc-1.0.tar.gz.

File metadata

Download URL: msgq_ipc-1.0.tar.gz
Upload date: Sep 6, 2026
Size: 35.5 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for msgq_ipc-1.0.tar.gz
Algorithm	Hash digest
SHA256	`4c6407adea169d752e11192f2af193ddd39f89fca031fc84d6d284fbc829c8f9`
MD5	`d3f1e4764a91bb45a2d8efac96a5438a`
BLAKE2b-256	`3512058fbf562eefdcb53a38420748bbeadbc321e4c33975f1ce882598f1d930`

Algorithm	Hash digest
SHA256	`4526bbe9522a4ee334c105b394fe2f57f4097ec5a65d0df4338086f947dc4825`
MD5	`0a50259293281ce273d6b3924f19ac44`
BLAKE2b-256	`c5f97c81dcf108ba58aa56c9a51ebca4b1c75c05d6ed5fd168eeab941a502bb5`

Algorithm	Hash digest
SHA256	`c3941b741e26a487c4fb65d4fe0ba4141e7ab4c624d5f4cc597888a06e5224f5`
MD5	`75aeeab6759a73572367f0d62ebe5dd4`
BLAKE2b-256	`dc19ed4524155cdbbc5d337d59c63bce369e54e8111352bd4d43c979727f33e8`

Algorithm	Hash digest
SHA256	`53c3718a956f5227bb3471bb0709a7109af43631a7ad1a5e5fcef975bc340e1b`
MD5	`0f5364a6b96f4595efaf0241bbc26879`
BLAKE2b-256	`baf458465bf84970edf6483f1a7217113e5ee07cfc980faad8f74dda54c80915`

Algorithm	Hash digest
SHA256	`362edb8974651418ffc05a487303b5af3691b10e638692d4273d5e3a373d9759`
MD5	`474d3ec92f9caa448b4daf9eb1c90089`
BLAKE2b-256	`e0c11467e2f25057a46c859b7a55ba6301fc53ec9dc75ff8fc18226f5d84ece8`

Algorithm	Hash digest
SHA256	`0d00ce9cbfeb5b1d3af1523a6a3cdfd471b2f37f375d3ab63b6463b987b2dec2`
MD5	`7467ca3e361d1046c1d75b744cea6596`
BLAKE2b-256	`cdadfef81acd30b2e7d52d4a9cb721937514663b4212eb909b2a009715136710`

Algorithm	Hash digest
SHA256	`cf0f3f2df0586b0216a3f6dc865a8c151f675bead7f83191c1d5ba09ab5fa409`
MD5	`2ddca7aba3d3664e068c7fd034aea910`
BLAKE2b-256	`4dcc932175b930fa71f959b1c7f49987d31b630e0068ea79641d4c3976001673`

Algorithm	Hash digest
SHA256	`e7090309e7985723a035fd7e2f8c1c7af584881fb31a50e539e8deb959bc0d6e`
MD5	`aad1568f9a97a5c88b764ad8de2f5f78`
BLAKE2b-256	`cebd16618499b016572ac17cdca8d5ccc5ac896201308a91fb30b6f03cdbb4cf`

Algorithm	Hash digest
SHA256	`986ea0b4321a4c71b8ae14a5731df2f7e09815ea20febe2e7d7be3d04cd4b775`
MD5	`33f2d8b7437573ebce550a2a027b8885`
BLAKE2b-256	`8dc9c29d7225fff9afc669c2202c9d8eac7d8928a52bfaa5fda1169fa006a460`

Algorithm	Hash digest
SHA256	`58bea17d1f6b983fd03edbf82b5aed04c3fbcdeda02f2520e7be402cc08c5667`
MD5	`bdee16753a2be942c05c1b2d7ce11a79`
BLAKE2b-256	`95c2d70bee7c0acfbc01ac26aeeaf4a7bfc7628844d1370922cdf17522c52ffa`

Algorithm	Hash digest
SHA256	`0de54e1190ce469f47d1603a80e6e0a22eabb09502d24df65847e4b5aece0a52`
MD5	`5e6c84974954e153efdbe9fae379b0c2`
BLAKE2b-256	`fd8cd8f8db5ba2804d0deefdcdf883b9e35e3aa793788ec1e0d7b2f60c6cc1bf`

Algorithm	Hash digest
SHA256	`c911e09fb0cba01a5752c4e86246338dd78edfe23ac260bdb904407a6f8b6759`
MD5	`161ef4688b4ed1056928b2b4a3cc59c7`
BLAKE2b-256	`7e245733dbeafe862c23666a3ca8e27af5d3881982d9b5754df58d88fdffaf48`

Algorithm	Hash digest
SHA256	`fe43cc5c80a5518524bcff883fed7773f7b0b32073705d5d251bb32f5600ab82`
MD5	`1d05e4b1558fb50fe1ac582628ea9709`
BLAKE2b-256	`5e8dfce6252d538a8299bc2d8e160a9727c7e0b935be61ec7b298f87384c6210`

Algorithm	Hash digest
SHA256	`b1be781e4abf379705b3bce4538d3016acbb91388014bc88c25070455ee5fc56`
MD5	`d824fde74aa667b6ed365c64a93de8fe`
BLAKE2b-256	`b8f7debbcdb6cf78c08ce7e79b6af46a227fff1bd4530f8c2fb12e3e47fe08c9`

Algorithm	Hash digest
SHA256	`2ff833c522d8be909337572bec91875399a5cf8ed23c21e9fb2af9b3979d1f1b`
MD5	`5a28a696c03d6a5c7962d34ff202f26c`
BLAKE2b-256	`690f9314e043f660962bce76e15a1faf3aab537ef212b584ec8887f0828d95af`

Algorithm	Hash digest
SHA256	`5fae798f2047756998e21ebe15e1e2097bb5a7c7a6299621272cef1a4ae4312f`
MD5	`c865db00be528a916f2409e115ea5c18`
BLAKE2b-256	`e54f859bf3094c8d4aab60be144f58080aba7a2f404deb43f6befd48eb2b4513`

Algorithm	Hash digest
SHA256	`c93ab0e1dfa20369473e311d7b19cc70c85c41f50fd2617e52b9f01da86fd393`
MD5	`121faf2c27b8513035f607ba1225bf2a`
BLAKE2b-256	`a682ecc7cd0c79e68a5b97909718b503b43c3a2a68d83f910f55362d2f7c840b`

Algorithm	Hash digest
SHA256	`3754b9abb6f895fcd28bc41675ba81c739fdd7375d0d30bf70baa75d9d130aed`
MD5	`0deda3ddbd3562328c03b0da23081c72`
BLAKE2b-256	`281a8c818a9ff1d55656e863dfdb3d3ef8b31549a43e7db640390320da996ae6`

Algorithm	Hash digest
SHA256	`46fcb4c87ed686540d49a91a800b61b12cc55e9398b82dad2a289c4f758ef8a6`
MD5	`d27b097fa63ef6c92396336aa43fd2bf`
BLAKE2b-256	`71e479ccf4527749ed278bb0632015aeb28e7574c24a5692258f24ab2b280526`

Algorithm	Hash digest
SHA256	`f6f7bf3a24aa4390aa0d091aa4df58e4d9a9045e219d637ba5e022ad7ba56fe1`
MD5	`4bc9653d956a58d6b2042486ff194d53`
BLAKE2b-256	`f847408161218eed5760bc7c756474227445b829288c9504c240856e93ede175`

Algorithm	Hash digest
SHA256	`d77b613913a7daa62b76b29d3264d797a7a3638e3db3d685d259899725ce29d2`
MD5	`1f0ce4702ba5dd508ee55a882943090b`
BLAKE2b-256	`a218e9f7a5379ab8b7092f7fd14a84264881a77b7dd8238678d6bd59086c90bc`

msgq-ipc 1.0

MSGQ

Quickstart · Examples · Discord

Quickstart

Contributing

License

Under the hood

Storage

Writing

Reset reader

Reading

Project links

Key dates

1 maintainer

Credits

License expression

Requires

Provides Extra

Classifiers

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

Algorithm	Hash digest
SHA256	`7a4f0c58bd1c529166bf3d029cddb55120e322a1b35523412f2e2d32e3d806a7`
MD5	`dc8bb5db07602fe11816dfe168133be7`
BLAKE2b-256	`ce7d563e5b15a99fd3a216ff5b2a0aad87df16c5503dd2e2e44c1bf17197c0ff`

Algorithm	Hash digest
SHA256	`826d45f66a01e7a68e900816cfab919216d2ba936d439fbb6a738b7c1385a8f4`
MD5	`3bab70a5e002503682e916c31605fa05`
BLAKE2b-256	`4cee491bec42794e04b980569d9261e5709bb1e8d625ff39947b3010ab7f14d8`

Algorithm	Hash digest
SHA256	`0360e8ab511e2e52c76054b5ea3a9aaaf19c3b4aa944ac2e3e86ed041fb4f964`
MD5	`06a901ede1f8e4b2982c64e44dc4e8bb`
BLAKE2b-256	`2e9e77cec6b42cf416b4b56bef5f0a044aaaee37bc939219aa9083fafe2f2d5f`

Algorithm	Hash digest
SHA256	`ee521966ac82d3742632ddebffb1788cdeea0af4bc5584201a3144338ea959be`
MD5	`7b861f62eb811d67bf6ad39fc1b50563`
BLAKE2b-256	`5f8b0ac2fce79f2cc030ffdc8f5856dde1c4df4578e3acfb1a5d252814784f1e`