A thread-safe SPSC queue, implemented on a ring buffer written in C
Project description
spsc-ring-threadsafe
Lockless, thread-safe, single-producer, single-consumer, FIFO queue for Python — implemented on a ring buffer in C.
Design
Built on a Lamport ring buffer with C11 _Atomic read/write indexes (acquire/release ordering). Producer and consumer never contend with eachother.
Features:
- single producer, single consumer, FIFO semantics
- lockless (no mutexes or spinlocks)
- non-blocking / async-friendly
- suitable for shared memory and IPC
- compatible with no-GIL & subinterpreters
- low overhead, especially for small messages
- up to 100x faster than
multiprocessing.Queuein the standard library
Benchmarks
Quick Start
import spsc_ring_threadsafe as srt
buf = bytearray(4096)
srt.init(buf) # initialize as ring buffer
item = b"this is a bytestring message"
srt.put(buf, item)
result = srt.get(buf)
print(result.decode())
Output:
this is a bytestring message
API Reference
Exceptions
| Exception | Raised when |
|---|---|
spsc_ring_threadsafe.QueueFullError |
put() called on a full buffer |
spsc_ring_threadsafe.QueueEmptyError |
get() called on an empty buffer |
Functions
spsc_ring_threadsafe.init(buf)
Initialize a mutable buffer as a ring buffer (this function sets the read + write indexes to zero).
| Parameter | Description |
|---|---|
buf |
Mutable buffer-compatible object (bytearray, memoryview, etc.). Size must be a power-of-two anywhere from 256 bytes to 2 GiB. Also accepts shared memory buffers. |
⚠️ Thread safety: Initialization is not thread-safe. Initialize once before any concurrent access, or provide your own synchronization.
spsc_ring_threadsafe.put(buf, item)
Insert an item into the ring buffer. Non-blocking.
| Parameter | Description |
|---|---|
buf |
Mutable buffer-compatible object. Size must be a power-of-two anywhere from 256 bytes to 2 GiB. |
item |
Buffer-compatible object to insert |
Raises: QueueFullError if the buffer has insufficient space.
⚠️ Buffer must be zeroed or initialized before use. ⚠️ Only a single producer is allowed for a given queue! Multiple producers are NOT thread-safe.
spsc_ring_threadsafe.get(buf)
Remove and return an item from the ring buffer. Non-blocking.
| Parameter | Description |
|---|---|
buf |
Mutable buffer-compatible object. Size must be a power-of-two anywhere from 256 bytes to 2 GiB. |
Returns: Buffer-compatible object containing the message.
Raises: QueueEmptyError if no message is available.
⚠️ Buffer must be zeroed or initialized before use. ⚠️ Only a single consumer is allowed for a given queue! Multiple consumers are NOT thread-safe.
Installation
uv add spsc-ring-threadsafe
or
pip install spsc-ring-threadsafe
Development
Run tests
uv run python -m unittest discover -s tests -v
Run benchmarks
uv pip install -e . --force-reinstall --no-deps
uv run python tests/benchmark.py
Run dev script
uv pip install -e . --force-reinstall --no-deps
uv run python tests/dev.py
LLM Usage Disclosure
Tests and benchmarks were written with assistance from AI models. All C code is written by hand.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file spsc_ring_threadsafe-0.0.8-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.
File metadata
- Download URL: spsc_ring_threadsafe-0.0.8-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
- Upload date:
- Size: 18.1 kB
- Tags: CPython 3.14, manylinux: glibc 2.17+ x86-64, manylinux: glibc 2.28+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.9.13 {"installer":{"name":"uv","version":"0.9.13"},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Linux Mint","version":"22.3","id":"zena","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e9ebb94cfc59d364c4c52f7ad5858913917ca3b2e7f8f78e4a4ba589d78f3e5b
|
|
| MD5 |
0fbdc9d66532a232dde8e2cce4db246f
|
|
| BLAKE2b-256 |
f4cf647a9583caa12318b278d1970c4c11f874142aa6b0f0498d791d423981da
|