Skip to main content


Project description

Master: travis-master coveralls-master

Develop: travis-develop coveralls-develop

Concurrency utilities for Python 3.7 + asyncio; the main portion is a component abstraction. To support this, some pipe implementations for inter-task, inter-thread, and inter-process communication and some serialization utilities are provided as well. Inter-process communication is based on ZeroMQ. Finally a class EventLoopThread is provided that can be used for bridging synchronous and asynchronous code.

A “component” is code that is executing on its own, like an asyncio task, a thread, a worker thread’s load, or a process. Components process commands issued by their owner, and create events to be handler by their owner. Components may also produce a result, and of course may communicate with other entities than their owner.

Although asyncio is used heavily, the connection between a workload and its owner decouples the two to allow for any model of concurrency. Here is an example, taken and adapted from the test suite:

import asyncio
from concurrent.futures import ThreadPoolExecutor

from concurrent_utils.component import Component, component_workload, start_component_in_thread

async def test_thread_component_result_success_and_command():
    async def component(x, *, commands, events):
        await events.send(Component.EVENT_START)
        ### startup complete

        # reply to command
        await commands.send(await commands.recv() + 1)

        # return
        return x

    e = ThreadPoolExecutor(1)
    comp = await start_component_in_thread(e, component, 1)

    assert await comp.request(1) == 2

    assert await comp.result() == 1

Although the component is defined as a coroutine, it is (in this case) not executed on the owner’s event loop. The same component could, without modification, be run on the owner’s event loop, or, with minor modifications to make the component function pickle-able, be run in a worker process instead.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for concurrent-utils, version 0.2.0
Filename, size File type Python version Upload date Hashes
Filename, size concurrent_utils-0.2.0-py3-none-any.whl (25.0 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size concurrent-utils-0.2.0.tar.gz (24.8 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page