Thread-safe, mixed sync-async queue for Python
Project description
newt
Thread-safe, mixed-threading-and-asyncio, producer-consumer queue for Python.
Heavily based on janus, but newt lazily initializes event loop which makes the queue much more flexible:
newt.Queue
could be initialized outside subthread or coroutine- supports information exchange between a thread and a coroutine
- ensures thread-safety
Install
$ pip install newt
Usage
Suppose there is a threaded target function which produces items, and a coroutine which consumes items.
from newt import Queue
def threaded(sync_queue):
for i in range(100):
sync_queue.put(i)
sync_queue.join()
sync_queue
follows the interface of Python built-in synchronized queue class
async def coroutine(async_queue):
for i in range(100):
assert await async_queue.get() == i
async_queue.task_done()
async_queue
follows the vanilla Python asyncio.Queue
Thread in an executor -> Coroutine
The following example shows how to produce item in a thread which executed in the executor, and consume the item in a coroutine.
import asyncio
loop = asyncio.get_event_loop()
async def main():
future = loop.run_in_executor(None, threaded, queue.sync_queue)
await coroutine(queue.async_queue)
await future
queue.close()
await queue.wait_closed()
loop.run_until_complete(main())
Normal thread -> Coroutine
newt.Queue
also supports to produce item in a normal threading,
loop = asyncio.get_event_loop()
async def main():
await coroutine(queue.async_queue)
queue.close()
await queue.wait_closed()
t = threading.Thread(target=threaded, args=(queue.sync_queue,))
t.start()
loop.run_until_complete(main())
License
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file newt-0.1.0.tar.gz
.
File metadata
- Download URL: newt-0.1.0.tar.gz
- Upload date:
- Size: 9.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.3.1 requests-toolbelt/0.9.1 tqdm/4.46.1 CPython/3.8.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ee184e66764b9ec3f6fae548f30d5c73a33a7b213a7058523623967d1ce55c64 |
|
MD5 | 12926af78d3ce6a7c3525745a7176119 |
|
BLAKE2b-256 | 82cf64f2db5fa2c218ddf3614d5298a5a2a2c7a58091ccd1f4c29f0fcd6888db |
File details
Details for the file newt-0.1.0-py3-none-any.whl
.
File metadata
- Download URL: newt-0.1.0-py3-none-any.whl
- Upload date:
- Size: 9.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.3.1 requests-toolbelt/0.9.1 tqdm/4.46.1 CPython/3.8.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 76910335ef3daef346e694e428e88f26bb64f0f85c3907204c02f3121b088ec7 |
|
MD5 | 78ca14746550633ebf67427a636e9ad4 |
|
BLAKE2b-256 | 2aa0d3b2c47ad22713d5772cc45ee184517a2adee78794e00524b47485d64c85 |