Skip to main content

Simpler multiprocessing framework

Project description

Today’s computers are almost always multicore machines. However, due to CPython’s GIL, exploiting these multiple cores are not simple. The multithreading module, for instance, are practically rendered impotent as multithreading won’t happen unless a thread is being tied up in an I/O operation.

However, Python provides a way to actually harness multicore parallelism: The multiprocessing module.

The problem with multiprocessing is that you are given a very ‘low-level’ plumbing mechanism. It provides you with tools to create multiple (sub-)processes, tools to communicate between the (sub-)processes, tools to synchronize between the (sub-)processes… but that’s about it. When you start making programs to harness multiprocessing’s powers, you’ll quickly find out that you have to keep building the exact same code structure. And along the way, you keep ‘re-creating the wheel’, so to speak.

This package aims to provide Python programmers with a ‘slightly higher level’ view of multiprocessing, by removing the necessity of keep having to create the ‘harness’ code.

In a nutshell, this code aims to be to multiprocessing what requests did for urllib: Abstracting all the plumbing and providing a more intuitive toolset to implement multicore-aware programs.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for multiworker, version 0.1.1a0.dev1
Filename, size File type Python version Upload date Hashes
Filename, size (3.7 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page