Nonblocking stream reads using Queue and a pump thread
Project description
Nonblocking Stream Queue
This is a simple python package for nonblocking reading from streams.
It could be expanded for general nonblocking usage if desired.
A simple nonblocking queue reader is provided, with implementation from https://stackoverflow.com/questions/375427/a-non-blocking-read-on-a-subprocess-pipe-in-python/4896288#4896288 .
When constructed, the reader spawns a thread and reads everything from the stream, until garbage collection or EOF.
Usage
import sys
import nonblocking_stream_queue as nonblocking
reader = nonblocking.Reader(
sys.stdin,
max_size=4096,
lines=False,
max_count=None,
drop_timeout=None,
transform_cb=None
)
print(reader.read_one()) # outputs up to 4096 characters, or None if nothing queued
print(reader.read_many()) # outputs all 4096 character chunks queued
print(''.join(reader.read_many())) # outputs all queued text
reader.block() # waits for data to be present or end, returns number of reads queued
reader.is_pumping() # False if data has ended
Timestamping
import sys, time
import nonblocking_stream_queue as nonblocking
reader = nonblocking.Reader(
sys.stdin.buffer,
transform_cb=lambda data: (time.time(), data)
)
while reader.block():
print(*reader.read_one()) # shows the timestamp each item was buffered
Lines
import sys
import nonblocking_stream_queue as nonblocking
reader = nonblocking.Reader(
sys.stdin,
lines=True
)
while reader.block():
print(reader.read_one().rstrip()) # outputs each line of text that is input
Other solutions
There are likely many other existing solutions to this common task.
Here's one:
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for nonblocking_stream_queue-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 15cba0eea1e9124f5458d5390ea43d81d247deb653013e97663ba5357653414d |
|
MD5 | 0aee095ad165efc59648373ca1c7fe70 |
|
BLAKE2b-256 | 5934ee5d18fe5e9917994d0924f9f414552b4dce111e77ab6fb89d37a2f7c2f7 |
Close
Hashes for nonblocking_stream_queue-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c2cb9df55ddc85a5e8d43a3b894b5a486e62e7252c5560e1d6880271288c9d96 |
|
MD5 | 2cc7f655c9c1a8ad8feb9af2290cde26 |
|
BLAKE2b-256 | de4ca8e32cd28b86ad72d7d1d3ed7973ba379a7391a39b8cc8433c85da657af0 |