Skip to main content

Multiprocessing queue for cross-platform development.

Project description

The 'EnhaaancedQueues' library contains the 'EnhQueue' class - coming with extended / enhanced (in particular portability) capabilities.

The 'EnhQueue' class is inherited from 'multiprocessing.queues.JoinableQueue' - which after that is enhanced to work properly on all main desktop platforms ('Windows', 'macOS' and 'Linux').

Windows:

Windows processes do not share memory, which means, shared objects have to be 'pickled' and transmitted to the respective subprocess via pipe - when given as parameters to (multiprocessing) process calls. The 'EnhQueue' class has been made pickable (have a look at it's '__ reduce __' method).

macOS:

The 'qsize' method of the 'multiprocessing.queues.JoinableQueue' does not work under some 'macOS'es (and maybe other *nixes); it might lead to a 'NotImplementedError'.

The 'EnhQueue' class wraps 'multiprocessing.queues.JoinableQueue' to correct this aspect. The additional lock used for that might also have a positive effect to the reliability of the methods 'empty' and 'full'.

Miscellaneous:

Shutting down an application with several processes properly is a delicate thing, if some of said processes crashed / got stuck. The 'EnhQueue' class has been equipped with a semi-blocking 'join' method - a 'join' with a timeout.

Due to the parallel nature of multiprocessing, emptying a queue reliable in a setup, where several processes could put another element in said queue during or after said emptying is a delicate thing too.

The 'EnhQueue' has been equipped with a 'clear' method emptying the queue in the most reliable and least blocking way possible - as known to the author yet. It in particular is designed to be useful during cleaning up when shutting down an application properly.


Further infomations and links can be found on my homepage https://www.blackward.de

Have Fun!

Project details


Release history Release notifications | RSS feed

This version

0.7

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

enhaaancedQueues-0.7-py2.py3-none-any.whl (8.3 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file enhaaancedQueues-0.7-py2.py3-none-any.whl.

File metadata

  • Download URL: enhaaancedQueues-0.7-py2.py3-none-any.whl
  • Upload date:
  • Size: 8.3 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.1 pkginfo/1.7.1 requests/2.22.0 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.8.10

File hashes

Hashes for enhaaancedQueues-0.7-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 d4c664eec5c9202826331b13a8294fad2b6007cb3993faceb72a8888fe333f5c
MD5 ee736d61333c64d2abacf2df52d5c587
BLAKE2b-256 c7a022a3fccb79b39d708c98b2716ff03042270b73ebd501af9e69173c3ef1e9

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page