A modern queue in a multithreaded environment
Project description
ModernQueue
A modern and permissive Python queue in a multithreaded environment.
Go to the Wiki for more information.
Table of contents
Installation
From PyPI
pip install modernqueue
From GitHub
Download modernqueue.py
and put it in your project folder.
Usage
Importing
from modernqueue import ModernQueue
Creating a queue
# Create a queue with 4 threads
queue = ModernQueue(max_threads=4)
# Create a queue with no limit on threads
queue = ModernQueue()
Adding functions to the queue
# Add a function to the queue
queue.add(func=print_number, args={'number': 1})
# Add a function to the queue, with tuple arguments
queue.add(func=print_number, args=(1,))
Running the queue
# Run the queue, blocking the function until finished
queue.run()
# Run the queue, without blocking the function
queue.run(is_blocking=False)
Getting the results
# Get the results of the queue, in order
results = queue.get_results()
# Get faster the results of the queue
results = queue.get_results(is_ordered=False)
Waiting for the queue to finish
# Wait for the queue to finish, in blocking mode
queue.run()
# Wait for the queue to finish, in non-blocking mode
queue.run(is_blocking=False)
while queue.running() != 0:
sleep(0.1)
Examples
1)
Simple example of a queue with 4 threads, that prints numbers from 1 to 10, with a 1-second delay between each number.
This one is greatly commented, so you can understand how it works.
# Define the function to run
def print_number(number: int) -> int:
"""
Print a number and sleep for 1 second.
Args:
- number (int): The number to print.
Returns:
- (int) The number multiplied by 2.
"""
sleep(1)
print(number)
return number * 2
# Create the queue, with a maximum of 4 threads
#
# max_threads is optional and defaults to -1 (no limit)
queue = ModernQueue(max_threads=4)
# Add the functions to the queue
for i in range(1, 11):
# There are 2 ways to pass arguments to the function
# 1. As a dict (kwargs):
queue.add(func=print_number, args={'number': i})
# 2. As a tuple (args):
# queue.add(func=print_number, args=(i,))
# Run the queue, blocking the function until finished
# is_blocking is optional and defaults to True
queue.run(is_blocking=True)
# Print "Done", if the function is blocking
# This will be printed after all the numbers are printed
print("Done")
# Get the results of the queue
#
# If you don't want to take the processing time to sort the results,
# set is_ordered to False
#
# is_ordered is optional and defaults to True
results = queue.get_results(is_ordered=True)
print(results)
# --- OUTPUT ---
# 4
# 2
# 3
# 1
# 5
# 7
# 6
# 8
# 9
# 10
# Done
# [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
2)
Same as above, but with fewer comments.
# waiting for the queue to finish in blocking mode
queue = ModernQueue(max_threads=4)
for i in range(1, 11):
queue.add(func=print_number, args={'number': i})
queue.run(is_blocking=True)
print("Done")
results = queue.get_results(is_ordered=True)
print(results)
# --- OUTPUT ---
# 1
# 3
# 2
# 4
# 5
# 7
# 6
# 8
# 9
# 10
# Done
# [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
3)
Same as above, but in non-blocking mode.
It will print “Not done yet…” while the queue is running, and “Done” when it's finished. To mark when the process is finished and when it's not in the code.
# waiting for the queue to finish in non-blocking mode
queue = ModernQueue(max_threads=4)
for i in range(1, 11):
queue.add(func=print_number, args={'number': i})
queue.run(is_blocking=False)
print("Not done yet...", f"({queue.running()} threads running)")
while queue.running() != 0:
sleep(0.1)
print("Done")
results = queue.get_results(is_ordered=True)
print(results)
# --- OUTPUT ---
# 1
# 3
# 2
# 4
# 5
# 7
# 6
# 8
# Not done yet... (2 threads running)
# 9
# 10
# Done
# [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
License
You can use this code however you want, no credit is required.
Though, if you want to give me credit, you can do it by linking to my GitHub profile: https://github.com/PonyLucky
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for modernqueue-1.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | abd04f7dedf82f7bfda1fa65bbdd7549cde1aa267f358e9f009a16fdcb955591 |
|
MD5 | 763716c77d57f3cadd0aaeaa3df38599 |
|
BLAKE2b-256 | a2818054c3f47016c939ddf2aa4eec83f64205b6aebef3aee6856f29cd07b210 |