cpu_count
Modified version of python's cpu_count that takes into account system
constraints to calculate the number of available CPUs
Motivation
The Python standard library offers an implementation of cpu_count that returns
the real number of CPUs even when they are not actually available to be used by
the python process (due to constraints such as CPU affinity or CPU scheduler
configurations). This is the preferred behaviour for most applications. However,
when the interest is the amount of CPUs available for data processing, that
approach could be misleading. Especially when it is used behind the scenes such
as by the concurrent.futures.Executor when defining its defaults.
The purpose of this module is to provide this functionality in an API equal to the standard implementation. By taking into account the described constraints, this implementation attempts to return the amount of usable CPUs that are available. If no constraint is identified the result will be the same as the standard implementation.
How to install
pip install cpu_count
How to use
As an external module
This is the standard way. Just import and call cpu_count
from cpu_count import cpu_count
print(cpu_count())
# $> 8
Monkey-patch standard lib
This an alternative way that replaces python's standard cpu_count with the
one from this module (Affected internal modules are posix, os and
multiprocessing). The advantage of this approach is not needing to port any
code, just import and call setup_monkey_patch ate the begin of your
application and everything will just work™.
Note: This will also have implications in the behaviour of standard libraries that use this function
import os
from cpu_count.monkey_patch import setup_monkey_patch
print(os.cpu_count())
# $> 12
setup_monkey_patch()
print(os.cpu_count())
# $> 8
Limitation
This approach has one limitation: it can't replace previous code that
imported the standard implementation using from os import cpu_count.
from os import cpu_count
from cpu_count.monkey_patch import setup_monkey_patch
print(cpu_count())
# $> 12
setup_monkey_patch()
print(cpu_count())
# $> 12
System wide monkey-patch
This approach also replaces python's standard cpu_count. However instead of
calling this module's setup_monkey_patch in your application code, it will be
called at python startup. For this to work you need to create a file called
cpu_count_monkey_patch.pth at your python's global or local site-package's
folder with the following content:
import cpu_count; cpu_count.monkey_patch.setup_monkey_patch()
NOTE: This approach is specially useful when creating container images of python applications. An example of using this on a Dockerfile can be found here
TODO
- Add logic for Realtime Scheduler constraint
- Create unit tests
Contributions
All contributions are very welcome.
Code styles is defined in the Editorconfig file. Besides that I use black and isort for auto-format, their configurations are defined in the Editorconfig and the black.toml files respectively.
License
BSD 3-Clause “New” or “Revised” License
See LICENSE
Metadata
Release files for cpu-count 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| cpu_count-1.0.0.tar.gz | 6.2 kB | Without provenance | Details for cpu_count-1.0.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| cpu_count-1.0.0-py3-none-any.whl | Python 3 | none | any | Details for cpu_count-1.0.0-py3-none-any.whl |
Total release size: 13.6 kB
Release files / cpu_count-1.0.0.tar.gz
| Download URL | cpu_count-1.0.0.tar.gz |
|---|---|
| Size | 6.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9edd8796a7fd14143f821acd4b4fc7f8e933fe9c064805892137ff9ff72a5f8b
|
|
BLAKE2b-256 checksum How to use checksums |
2ee3dc921e26a47b7c549795e934008b9683f0bd1ddd6aa1a08fbc2825d25822
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.7.4
|
Release files / cpu_count-1.0.0-py3-none-any.whl
| Download URL | cpu_count-1.0.0-py3-none-any.whl |
|---|---|
| Size | 7.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
7cfc4130d0e44b4bee357b96d4479010d9a9fed68fb3e74ba8e87c549c0fab88
|
|
BLAKE2b-256 checksum How to use checksums |
b52057fd25f1a9bd90a56a75fe1976d0c147d7a5167a2baf28a85ba9afa961cf
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.7.4
|
Security for cpu-count 1.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.