This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Async tools for Python

Project Description

AsyncTools

Async Tools for Python.

Table of Contents

  • Threading
    • Async
    • Parallel
    • Pool

Threading

Threading is the most simple thing, but because of GIL it’s useless for computation. Only use when you want to parallelize the access to a blocking resource, e.g. network.

Async

Source: asynctools/threading/Async.py

Decorator for functions that should be run in a separate thread. When the function is called, it returns a `threading.Event <https://docs.python.org/2/library/threading.html#event-objects>`__.

from asynctools.threading import Async

@Async
def request(url):
    # ... do request

request('http://example.com')  # Async request
request('http://example.com').wait()  # wait for it to complete

If you want to wait for multiple threads to complete, see next chapters.

Parallel

Source: asynctools/threading/Parallel.py

Execute functions in parallel and collect results. Each function is executed in its own thread, all threads exit immediately.

Methods:

  • __call__(*args, **kwargs): Add a job. Call the Parallel object so it calls the worker function with the same arguments
  • map(jobs): Convenience method to call the worker for every argument
  • first(timeout=None): Wait for a single result to be available, with an optional timeout in seconds. The result is returned as soon as it’s ready. If all threads fail with an error – None is returned.
  • join(): Wait for all tasks to be finished, and return two lists:
    • A list of results
    • A list of exceptions

Example:

from asynctools.threading import Parallel

def request(url):
    # ... do request
    return data

# Execute
pll = Parallel(request)
for url in links:
    pll(url)  # Starts a new thread


# Wait for the results
results, errors = pll.join()

Since the request method takes just one argument, this can be chained:

results, errors = Parallel(request).map(links).join()

Pool

Source: asynctools/threading/Pool.py

Create a pool of threads and execute work in it. Useful if you do want to launch a limited number of long-living threads.

Methods are same with `Parallel <#parallel>`__, with some additions:

  • __call__(*args, **kwargs)
  • map(jobs)
  • first(timeout=None)
  • close(): Terminate all threads. The pool is no more usable when closed.
  • __enter__, __exit__ context manager to be used with with statement

Example:

from asynctools.threading import Pool

def request(url):
    # ... do long request
    return data

# Make pool
pool = Pool(request, 5)

# Assign some job
for url in links:
    pll(url)  # Runs in a pool

# Wait for the results
results, errors = pll.join()
Release History

Release History

This version
History Node

0.1.2-2

History Node

0.1.2-1

History Node

0.1.2-0

History Node

0.1.0-0

History Node

0.0.1-0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
asynctools-0.1.2_2-py2-none-any.whl (7.9 kB) Copy SHA256 Checksum SHA256 2.7 Wheel Aug 14, 2014
asynctools-0.1.2-2.tar.gz (6.3 kB) Copy SHA256 Checksum SHA256 Source Aug 14, 2014

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting