Skip to main content

Threading and multiprocessing eye-candy.

Project description

Description

Pebble provides a neat API to manage threads and processes within an application.

Examples

Launch a task in a thread and wait for its results:

from pebble import thread


@thread
def do_job(foo, bar=0):
    return foo + bar


if __name__ == "__main__":
    task = do_job(1, bar=2)
    print task.get()  # it will block until do_job has completed

Launch five tasks in separate processes and handle their results in a callback:

from pebble import process


def task_done(task):
    print "Task %s has returned %d" % (task.id, task.get())


@process(callback=task_done)
def do_job(foo, bar=0):
    return foo + bar


if __name__ == "__main__":
    for i in range(0, 5):
        do_job(i)

    raw_input("Press return to exit.")

Callbacks can be dynamically (re)assigned, useful to set instance methods as callback:

import time
from pebble import process


class Foo(object):
    def __init__(self):
        self.counter = 0
        self.errors = 0
        self.do_job.callback = self.task_done

    def task_done(self, task):
        try:
            self.counter += task.get()
        except:  # exception are re-raised by the get() method
            self.errors += 1

    @process
    def do_job():
        return 1

    @process
    def do_wrong_job():
        raise Exception("Ops!")


if __name__ == "__main__":
    foo = Foo()
    tasks = []

    for i in range(0, 5):
        task = foo.do_job()
        tasks.append(task)
        task = foo.do_wrong_job()
        tasks.append(task)

    time.sleep(1)

    print foo.counter
    print foo.errors

Thread pools allow to execute several tasks asynchronously without the need of spawning a new thread for each task:

from threading import current_thread
from pebble import ThreadPool


def task_done(task):
    results, thread = task.get()
    print "Task %s has returned %d from thread %s" % (task.id, results, thread.ident)


def do_job(foo, bar=0):
    return foo + bar, current_thread()


if __name__ == "__main__":
    with ThreadPool(callback=task_done, workers=5) as tp:
        for i in range(0, 10):
            tp.schedule(do_job, args=(i, ), callback=task_done)

    raw_input("Press return to exit.")

TODO

A roadmap:

* pools of workers::

  - ProcessPool
  - @process_pool

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Pebble-2.2.0.tar.gz (7.9 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page