Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

A minimalist tool to run multiple parallel tasks in Python

Project Description

This is a minimalist tool to run multiple parallel tasks in python.

Most programming languages have full support for threads but often require a lot of overhead work even for the simplest tasks. This package aims to provide an easy way to parallelize these tasks with very little effort.

Getting Started

First install the package.

pip install mparallel

In your python modules, just import it and use it as follows:

import time
import mparallel

def some_expensive_or_waiting_task(some_param):
  # ...
  time.sleep(2)
  return some_param

def my_method():
  runner = mparallel.Runner()
  for i in range(10):
    runner.add_task(some_expensive_or_waiting_task, i)
  print runner.results()

You can see the tasks are being run in parallel from the previous code because even though they are being started in order (0..9), the final output will likely appear in a different order. Also, the total waiting time will be less than 20 seconds, which is the time it would take to serially run them.

Release History

Release History

This version
History Node

0.0.1

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting