A `ThreadedProcessPoolExecutor` is formed by a modified `ProcessPoolExecutor` that generates processes that use a `ThreadPoolExecutor` instance to run the given tasks.
Project description
The ThreadedProcessPoolExecutor class is an Executor subclass that uses a pool of process with an inner pool of threads on each process to execute calls asynchronously.
ThreadedProcessPoolExecutor is formed by a modified ProcessPoolExecutor that processes (with at most max_processes) that use a ThreadPoolExecutor instance (with at most max_threads) to run the given tasks.
If max_processes is None or not given, it will default to the number of processors on the machine.
If max_threads is None or not given, it will default to the number of processors on the machine, multiplied by 5.
Example
from concurrent.futures import as_completed import math from threadedprocess import ThreadedProcessPoolExecutor import requests RNGURL = "https://www.random.org/integers/?num=1&min=1&max=100000000&col=1&base=10&format=plain&rnd=new" def get_prime(): n = int(requests.get(RNGURL).text) if n % 2 == 0: return (n, False) sqrt_n = int(math.floor(math.sqrt(n))) for i in range(3, sqrt_n + 1, 2): if n % i == 0: return (n, False) return (n, True) with ThreadedProcessPoolExecutor(max_processes=4, max_threads=16) as executor: futures = [] for _ in range(128): futures.append(executor.submit(get_prime)) for future in as_completed(futures): print('%d is prime: %s' % future.result())
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Filename, size | File type | Python version | Upload date | Hashes |
---|---|---|---|---|
Filename, size threadedprocess-0.0.5.tar.gz (4.0 kB) | File type Source | Python version None | Upload date | Hashes View |