Skip to main content

Simpler multiprocessing framework

Project description

Today’s computers are almost always multicore machines. However, due to CPython’s GIL, exploiting these multiple cores are not simple. The multithreading module, for instance, are practically rendered impotent as multithreading won’t happen unless a thread is being tied up in an I/O operation.

However, Python provides a way to actually harness multicore parallelism: The multiprocessing module.

The problem with multiprocessing is that you are given a very ‘low-level’ plumbing mechanism. It provides you with tools to create multiple (sub-)processes, tools to communicate between the (sub-)processes, tools to synchronize between the (sub-)processes… but that’s about it. When you start making programs to harness multiprocessing’s powers, you’ll quickly find out that you have to keep building the exact same code structure. And along the way, you keep ‘re-creating the wheel’, so to speak.

This package aims to provide Python programmers with a ‘slightly higher level’ view of multiprocessing, by removing the necessity of keep having to create the ‘harness’ code.

In a nutshell, this code aims to be to multiprocessing what requests did for urllib: Abstracting all the plumbing and providing a more intuitive toolset to implement multicore-aware programs.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
multiworker-0.1.1a0.dev1.zip (3.7 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page