Skip to main content

Thread-pool Controls Build Status codecov

Python helpers to limit the number of threads used in the threadpool-backed of common native libraries used for scientific computing and data science (e.g. BLAS and OpenMP).

Fine control of the underlying thread-pool size can be useful in workloads that involve nested parallelism so as to mitigate oversubscription issues.

Installation

  • For users, install the last published version from PyPI:

    pip install threadpoolctl
    
  • For contributors, install from the source repository in developer mode:

    pip install -r dev-requirements.txt
    flit install --symlink
    

    then you run the tests with pytest:

    pytest
    

Usage

Command Line Interface

Get a JSON description of thread-pools initialized when importing python packages such as numpy or scipy for instance:

python -m threadpoolctl -i numpy scipy.linalg
[
  {
    "filepath": "/home/ogrisel/miniconda3/envs/tmp/lib/libmkl_rt.so",
    "prefix": "libmkl_rt",
    "user_api": "blas",
    "internal_api": "mkl",
    "version": "2019.0.4",
    "num_threads": 2,
    "threading_layer": "intel"
  },
  {
    "filepath": "/home/ogrisel/miniconda3/envs/tmp/lib/libiomp5.so",
    "prefix": "libiomp",
    "user_api": "openmp",
    "internal_api": "openmp",
    "version": null,
    "num_threads": 4
  }
]

The JSON information is written on STDOUT. If some of the packages are missing, a warning message is displayed on STDERR.

Python Runtime Programmatic Introspection

Introspect the current state of the threadpool-enabled runtime libraries that are loaded when importing Python packages:

>>> from threadpoolctl import threadpool_info
>>> from pprint import pprint
>>> pprint(threadpool_info())
[]

>>> import numpy
>>> pprint(threadpool_info())
[{'filepath': '/home/ogrisel/miniconda3/envs/tmp/lib/libmkl_rt.so',
  'internal_api': 'mkl',
  'num_threads': 2,
  'prefix': 'libmkl_rt',
  'threading_layer': 'intel',
  'user_api': 'blas',
  'version': '2019.0.4'},
 {'filepath': '/home/ogrisel/miniconda3/envs/tmp/lib/libiomp5.so',
  'internal_api': 'openmp',
  'num_threads': 4,
  'prefix': 'libiomp',
  'user_api': 'openmp',
  'version': None}]

>>> import xgboost
>>> pprint(threadpool_info())
[{'filepath': '/home/ogrisel/miniconda3/envs/tmp/lib/libmkl_rt.so',
  'internal_api': 'mkl',
  'num_threads': 2,
  'prefix': 'libmkl_rt',
  'threading_layer': 'intel',
  'user_api': 'blas',
  'version': '2019.0.4'},
 {'filepath': '/home/ogrisel/miniconda3/envs/tmp/lib/libiomp5.so',
  'internal_api': 'openmp',
  'num_threads': 4,
  'prefix': 'libiomp',
  'user_api': 'openmp',
  'version': None},
 {'filepath': '/home/ogrisel/miniconda3/envs/tmp/lib/libgomp.so.1.0.0',
  'internal_api': 'openmp',
  'num_threads': 4,
  'prefix': 'libgomp',
  'user_api': 'openmp',
  'version': None}]

In the above example, numpy was installed from the default anaconda channel and comes with the MKL and its Intel OpenMP (libiomp5) implementation while xgboost was installed from pypi.org and links against GNU OpenMP (libgomp) so both OpenMP runtimes are loaded in the same Python program.

Setting the Maximum Size of Thread-Pools

Control the number of threads used by the underlying runtime libraries in specific sections of your Python program:

from threadpoolctl import threadpool_limits
import numpy as np


with threadpool_limits(limits=1, user_api='blas'):
    # In this block, calls to blas implementation (like openblas or MKL)
    # will be limited to use only one thread. They can thus be used jointly
    # with thread-parallelism.
    a = np.random.randn(1000, 1000)
    a_squared = a @ a

Known Limitations

  • threadpool_limits can fail to limit the number of inner threads when nesting parallel loops managed by distinct OpenMP runtime implementations (for instance libgomp from GCC and libomp from clang/llvm or libiomp from ICC).

    See the test_openmp_nesting function in tests/test_threadpoolctl.py for an example. More information can be found at: https://github.com/jeremiedbb/Nested_OpenMP

    Note however that this problem does not happen when threadpool_limits is used to limit the number of threads used internally by BLAS calls that are themselves nested under OpenMP parallel loops. threadpool_limits works as expected, even if the inner BLAS implementation relies on a distinct OpenMP implementation.

  • Using Intel OpenMP (ICC) and LLVM OpenMP (clang) in the same Python program under Linux is known to cause problems. See the following guide for more details and workarounds: https://github.com/joblib/threadpoolctl/blob/master/multiple_openmp.md

Maintainers

To make a release:

Bump the version number (__version__) in threadpoolctl.py.

Build the distribution archives:

pip install flit
flit build

Check the contents of dist/.

If everything is fine, make a commit for the release, tag it, push the tag to github and then:

flit publish

Credits

The initial dynamic library introspection code was written by @anton-malakhov for the smp package available at https://github.com/IntelPython/smp .

threadpoolctl extends this for other operationg systems. Contrary to smp, threadpoolctl does not attempt to limit the size of Python multiprocessing pools (threads or processes) or set operating system-level CPU affinity constraints: threadpoolctl only interacts with native libraries via their public runtime APIs.

Metadata

Release files for threadpoolctl 2.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for threadpoolctl 2.1.0
File Size Uploaded Provenance Actions
threadpoolctl-2.1.0.tar.gz 26.6 kB Without provenance Details for threadpoolctl-2.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for threadpoolctl 2.1.0
File Interpreter ABI Platform Actions
threadpoolctl-2.1.0-py3-none-any.whl Python 3 none any Details for threadpoolctl-2.1.0-py3-none-any.whl

Total release size: 39.3 kB

Release files / threadpoolctl-2.1.0.tar.gz

Download URL threadpoolctl-2.1.0.tar.gz
Size 26.6 kB
Tags Source
SHA-256 checksum
How to use checksums
ddc57c96a38beb63db45d6c159b5ab07b6bced12c45a1f07b2b92f272aebfa6b
BLAKE2b-256 checksum
How to use checksums
78e8e39dc842f512ab5be11efe83160ddb7ad3c0cc1b8d42ce8c0469a0d2b926
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via python-requests/2.22.0

Release files / threadpoolctl-2.1.0-py3-none-any.whl

Download URL threadpoolctl-2.1.0-py3-none-any.whl
Size 12.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
38b74ca20ff3bb42caca8b00055111d74159ee95c4370882bbff2b93d24da725
BLAKE2b-256 checksum
How to use checksums
f712ec3f2e203afa394a149911729357aa48affc59c20e2c1c8297a60f33f133
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via python-requests/2.22.0

Security for threadpoolctl 2.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page