concurrent-collections 1.0.1

A brief description of concurrent-collections

Python Concurrent (thread-safe) collections

Thread-safe Python collections: ConcurrentBag, ConcurrentDictionary, and ConcurrentQueue.

tl;dr

Python's built-in list, dict, and deque are thread-safe for some operations, but not all.

concurrent_collections provides thread-safe alternatives by using locks internally to ensure safe concurrent access and mutation from multiple threads.

Why use these collections?

There is a lot of confusion on whether Python collections are thread-safe or not^{1, 2, 3}.

The bottom line is that Python's built-in collections are not fully thread-safe for all operations. While some simple operations (like list.append() or dict[key] = value) are thread-safe due to the Global Interpreter Lock (GIL), compound operations and iteration with mutation are not. This can lead to subtle bugs, race conditions, or even crashes in multi-threaded programs.

See the Python FAQ: "What kinds of global value mutation are thread-safe?" for details. The FAQ explains that only a handful of simple operations are guaranteed to be atomic and thread-safe. For anything more complex, you must use your own locking or a thread-safe collection.

I could not find any simple concurrent implementation, so concurrent_collections provides drop-in replacements that handle locking for you, making concurrent programming safer and easier.

1. Are lists thread-safe?

2. Google style guide advises against relying on Python's assignment atomicity

3. What kind of "thread safe" are deque's actually?

Installation

Pip:

pip install python-nameof

My recommendation is to always use uv instead of pip – I personally think it's the best package and environment manager for Python.

uv add python-nameof

Collections

ConcurrentBag

A thread-safe, list-like collection.

from concurrent_collections import ConcurrentBag

bag = ConcurrentBag([1, 2, 3])
bag.append(4)
print(list(bag))  # [1, 2, 3, 4]

ConcurrentDictionary

A thread-safe dictionary. For atomic compound updates, use update_atomic.

from concurrent_collections import ConcurrentDictionary

d = ConcurrentDictionary({'x': 1})
d['y'] = 2  # Simple assignment is thread-safe
# For atomic updates:
d.update_atomic('x', lambda v: v + 1)
print(d['x'])  # 2

ConcurrentQueue

A thread-safe double-ended queue.

from concurrent_collections import ConcurrentQueue

q = ConcurrentQueue()
q.append(1)
q.appendleft(0)
print(q.pop())      # 1
print(q.popleft())  # 0

License

MIT License