Skip to main content

A thread pool that auto-manages the number of worker threads and is compatible with the "futures" module ThreadPoolExecutor interface.

Project description


The CollapsingThreadPoolExecutor is inspired by and compatible with the ThreadPoolExecutor from the "futures" module, it operates differently in that worker threads are handled with a stack which results in the same worker or workers doing all the work (and idle workers being destroyed).

How to install

$ pip install collapsing-thread-pool-executor

How to develop


  • python3 w/ pip
  • python2 w/ pip
  • virtualenvwrapper
  • entr

Set up the environments

$ mkvirtualenv -p `which python2.7` collapsing-thread-pool-executor-py2
$ pip install .
$ pip install -r requirements.txt

$ mkvirtualenv -p `which python3` collapsing-thread-pool-executor-py3
$ pip install .
$ pip install -r requirements.txt

Watch the tests

# watch python2 tests in one window
$ workon collapsing-thread-pool-executor-py2
$ find ./ -name '*.py' | entr -c py.test -v --log-level=DEBUG collapsing_thread_pool_executor

# watch python3 tests in one window
$ workon collapsing-thread-pool-executor-py3
$ find ./ -name '*.py' | entr -c py.test -v --log-level=DEBUG collapsing_thread_pool_executor


The example below will execute some_task() 100 times; as some_task() should take a second to execute and as we've allocated 10 workers, the whole thing should take about 10 seconds.

import time

from collapsing_thread_pool_executor import CollapsingThreadPoolExecutor

def some_task():

# all arguments are optional
pool = CollapsingThreadPoolExecutor(

for i in range(0, 100):

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for collapsing-thread-pool-executor, version 2020.7.18
Filename, size File type Python version Upload date Hashes
Filename, size collapsing_thread_pool_executor-2020.7.18-py2-none-any.whl (8.0 kB) File type Wheel Python version py2 Upload date Hashes View
Filename, size collapsing-thread-pool-executor-2020.7.18.tar.gz (6.4 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page