Skip to main content

A `ThreadedProcessPoolExecutor` is formed by a modified `ProcessPoolExecutor` that generates processes that use a `ThreadPoolExecutor` instance to run the given tasks.

Project description

https://img.shields.io/pypi/v/threadedprocess.svg https://img.shields.io/pypi/l/threadedprocess.svg https://img.shields.io/pypi/pyversions/threadedprocess.svg https://travis-ci.org/nilp0inter/threadedprocess.svg?branch=master

The ThreadedProcessPoolExecutor class is an Executor subclass that uses a pool of process with an inner pool of threads on each process to execute calls asynchronously.

ThreadedProcessPoolExecutor is formed by a modified ProcessPoolExecutor that processes (with at most max_processes) that use a ThreadPoolExecutor instance (with at most max_threads) to run the given tasks.

If max_processes is None or not given, it will default to the number of processors on the machine.

If max_threads is None or not given, it will default to the number of processors on the machine, multiplied by 5.

Example

from concurrent.futures import as_completed
import math

from threadedprocess import ThreadedProcessPoolExecutor
import requests

RNGURL = "https://www.random.org/integers/?num=1&min=1&max=100000000&col=1&base=10&format=plain&rnd=new"


def get_prime():
    n = int(requests.get(RNGURL).text)

    if n % 2 == 0:
        return (n, False)

    sqrt_n = int(math.floor(math.sqrt(n)))
    for i in range(3, sqrt_n + 1, 2):
        if n % i == 0:
            return (n, False)
    return (n, True)


with ThreadedProcessPoolExecutor(max_processes=4, max_threads=16) as executor:
    futures = []

    for _ in range(128):
        futures.append(executor.submit(get_prime))

    for future in as_completed(futures):
        print('%d is prime: %s' % future.result())

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
threadedprocess-0.0.5.tar.gz (4.0 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page