Skip to main content

Multiprocessing queue for cross-platform development.

Project description

The 'EnhaaancedQueues' library contains the 'EnhQueue' class - coming with extended / enhanced (in particular portability) capabilities.

The 'EnhQueue' class is inherited from 'multiprocessing.queues.JoinableQueue' - which after that is enhanced to work properly on all main desktop platforms ('Windows', 'macOS' and 'Linux').

Windows:

Windows processes do not share memory, which means, shared objects have to be 'pickled' and transmitted to the respective subprocess via pipe - when given as parameters to (multiprocessing) process calls. The 'EnhQueue' class has been made pickable (have a look at it's '__ reduce __' method).

macOS:

The 'qsize' method of the 'multiprocessing.queues.JoinableQueue' does not work under some 'macOS'es (and maybe other *nixes); it might lead to a 'NotImplementedError'.

The 'EnhQueue' class wraps 'multiprocessing.queues.JoinableQueue' to correct this aspect. The additional lock used for that might also have a positive effect to the reliability of the methods 'empty' and 'full'.

Miscellaneous:

Shutting down an application with several processes properly is a delicate thing, if some of said processes crashed / got stuck. The 'EnhQueue' class has been equipped with a semi-blocking 'join' method - a 'join' with a timeout.

Due to the parallel nature of multiprocessing, emptying a queue reliable in a setup, where several processes could put another element in said queue during or after said emptying is a delicate thing too.

The 'EnhQueue' has been equipped with a 'clear' method emptying the queue in the most reliable and least blocking way possible - as known to the author yet. It in particular is designed to be useful during cleaning up when shutting down an application properly.


Further infomations and links can be found on my homepage https://www.blackward.de

Have Fun!

Project details


Release history Release notifications | RSS feed

This version

0.7

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

enhaaancedQueues-0.7-py2.py3-none-any.whl (8.3 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page