Skip to main content

concurrent processing foundations for Python: events, threads, and processes

Project description

Do you concur?

🏁 About

Warning

concur is a Work in Progress.

concur is set of modules for working with concurrent tasks. It attempts to provide a consistent programmer interface to scheduling such that tasks can be performed using threads or processes or events without changes to code. Naturally, there is some consistency degradation when using processes as memory boundaries are being crossed, but for common cases involving serializable objects, it can operate transparently.

Containment

Concur’s concur.libcontain module provides tools for working with the result of an arbitrary Python task, a callable. It’s a fundamental component for managing completion notifications as it allows you to refer to the exception as well as the return.

Contained returns are straightforward:

from concur import libcontain

def returns():
        return 5

c = libcontain.Contain(returns)
assert c.failed is False
assert c.open() == 5

However, contained raises are a little more amorphous:

class LocalException(Exception):
        pass

def raises():
        raise LocalException("error")

c = libcontain.Contain(raises)
assert c.failed is True
try:
        c.open()
except libcontain.Containment as err:
        assert isinstance(err.__cause__, LocalException)

Raises have to raise a new exception every time in order to avoid scribling on the original exception’s tracebacks. Advanced use of containment can raise the original cause, but such use requires care in order to avoid corrupting the traceback stack with frames from other raises.

Events

The concur.libevent module contains implementations for arbitrary joins and event driven queues, a callback deque.

Joins are devices that allow a set of endpoints to be defined such that all endpoints must be called in order for a commit to occur. The commit operation performs the final join, a call to another endpoint that recieves the mapping of endpoint identifiers to containers.

Delivery objects are a trivialization Deferred’s. Concur uses containment to pass error cases, so a more complication construct is not necessary.

Threads

The concur.libthread module primarily contains references to thread primitives provided by Python’s _thread module. However, it has an implementation of a deque-based Queue, and a simple Transition primitive for passing objects from thread to thread.

Processes

The concur.libprocess module contains primitives for working with isolated processes. Currently, support is limited.

Project details


Release history Release notifications

This version
History Node

0.4.0

History Node

0.2.0dev

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
concur-0.4.0.tar.gz (17.3 kB) Copy SHA256 hash SHA256 Source None Aug 15, 2012

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page