Skip to main content

Distibutes tasks among processes and threads.

Project description

What PT_Pool?

It's a combined process+threading pool. (having only "apply_async" method for now)

ThreadPoolExecutor on its own doesn't utilize more than one CPU core.
ProcessPoolExecutor on its own is not suitable to provide concurrency (e.g. slows down the system when large number of processes are created, and doesn't allow to create more than around 1000 processes on my system).

Installation

python3.7 -m pip install PT_Pool

Usage

from PT_Pool import PT_Pool
import time

def f(x):
    time.sleep(1)
    return x*x

def on_result(res):
    print(res)

pt_pool = PT_Pool(threads_limit = 100)

pt_pool.apply_async(f, range(10), on_result=on_result)

time.sleep(2)
input('Press enter to exit...')

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

PT_Pool-0.0.4.tar.gz (3.6 kB view hashes)

Uploaded Source

Built Distribution

PT_Pool-0.0.4-py3-none-any.whl (4.1 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page