paco 0.2.3

Small utility library for coroutine-based asynchronous generic programming

paco

Small and idiomatic utility library for coroutine-driven asynchronous generic programming in Python +3.4.

Built on top of asyncio, paco provides missing capabilities from Python stdlib in order to write asynchronous cooperative multitasking in a nice-ish way. Also, paco aims to port some of functools and itertools standard functions to the asynchronous world.

paco can be your utility belt to deal with asynchronous, I/O-bound, non-blocking concurrent code in a cleaner and idiomatic way.

Features

• Simple and idiomatic API, extending Python stdlib with async coroutines gotchas.

• Built-in configurable control-flow concurrency support (throttle).

• No fancy abstractions: it just works with the plain asynchronous coroutines.

• Useful iterables, decorators, functors and convenient helpers.

• Coroutine-based functional helpers: compose, throttle, partial, timeout, times, until, race…

• Asynchronous coroutines port of Python built-in functions: filter, map, dropwhile, filterfalse, reduce…

• Supports asynchronous iterables and generators (PEP0525)

• Concurrent iterables and higher-order functions.

• Better asyncio.gather() and asyncio.wait() with optional concurrency control and ordered results.

• Works with both async/await and yield from coroutines syntax.

• Reliable coroutine timeout limit handler via context manager.

• Designed for intensive I/O bound concurrent non-blocking tasks.

• Good interoperability with asyncio and Python stdlib functions.

• Composable pipelines of functors via | operator overloading.

• Small and dependency free.

• Compatible with Python +3.4.

Installation

Using pip package manager:

pip install --upgrade paco

Or install the latest sources from Github:

pip install -e git+git://github.com/h2non/paco.git#egg=paco

API

• paco.ConcurrentExecutor

• paco.apply

• paco.compose

• paco.concurrent

• paco.constant

• paco.curry

• paco.defer

• paco.dropwhile

• paco.each

• paco.every

• paco.filter

• paco.filterfalse

• paco.flat_map

• paco.gather

• paco.identity

• paco.interval

• paco.map

• paco.once

• paco.partial

• paco.race

• paco.reduce

• paco.repeat

• paco.run

• paco.series

• paco.some

• paco.throttle

• paco.thunk

• paco.timeout

• paco.TimeoutLimit

• paco.times

• paco.until

• paco.wait

• paco.whilst

• paco.wraps

Examples

Asynchronously and concurrently execute multiple HTTP requests.

import paco
import aiohttp

async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as res:
            return res

async def fetch_urls():
    urls = [
        'https://www.google.com',
        'https://www.yahoo.com',
        'https://www.bing.com',
        'https://www.baidu.com',
        'https://duckduckgo.com',
    ]

    # Map concurrent executor with concurrent limit of 3
    responses = await paco.map(fetch, urls, limit=3)

    for res in responses:
        print('Status:', res.status)

# Run in event loop
paco.run(fetch_urls())

Concurrent pipeline-style composition of transform functors over an iterable object.

import paco

async def filterer(x):
    return x < 8

async def mapper(x):
    return x * 2

async def drop(x):
    return x < 10

async def reducer(acc, x):
    return acc + x

async def task(numbers):
    return await (numbers
                   | paco.filter(filterer)
                   | paco.map(mapper)
                   | paco.dropwhile(drop)
                   | paco.reduce(reducer, initializer=0))

# Run in event loop
number = paco.run(task((1, 2, 3, 4, 5, 6, 7, 8, 9, 10)))
print('Number:', number) # => 36

License

MIT - Tomas Aparicio

History

v0.2.3 / 2018-10-23

• Merge pull request #42 from dylanjw/fix_syntax_error

• Use getattr to avoid async keyword

0.2.2 / 2018-10-09

• Merge pull request #40 from thatmattbone/master

• add loop param to paco.interval()

• fix(setup): use space based indentation

• fix(travis): use cpython 3.7-dev release

• refactor(errors): use “paco” prefix in exception messages

• chore(History): add version changes

v0.2.1 / 2018-03-21

• fix(#37): allow overload function signatures with variadic arguments

• refactor(timeout_test): remove print statement

• fix(docs): bad link to API reference

• refactor(docs): remove codesponsor

v0.2.0 / 2017-10-21

• refactor(api): API breaking change that modifies behavior by raising any legit exception generated by a coroutine.

• feat(examples): add examples file

• feat(v2): v2 pre-release, propagate raise exception if return_exceptions is False

• refactor(tests): add map error exception assertion test

• Merge branch ‘master’ of https://github.com/h2non/paco

• refactor(tests): add map error exception assertion test

• feat(docs): add sponsor banner

• feat(docs): add sponsor banner

• feat(LICENSE): update copyright year

• Update setup.py

v0.1.11 / 2017-01-28

• feat(api): add paco.interval function.

v0.1.10 / 2017-01-11

• fix(each.py,map.py): fixed return_exceptions kwarg

• fix(setup.py): add author email

• fix(Makefile): remove package file

v0.1.9 / 2017-01-06

• feat(api): add identity function

• feat(#31): add thunk function

• feat(package): add wheel package distribution

• refactor(wraps): simplify implementation

• fix(History): remove indentation

v0.1.8 / 2016-12-29

• feat(requirements): force upgrade setuptools

• feat(#29): support async iterable and generators

• fix(docs): link to examples

• chore(travis): use Python 3.6 stable release

0.1.7 / 2016-12-18

• feat(#26): add curry function.

0.1.6 / 2016-12-11

• feat(pipe): isolate pipe operator overload code

• refactor: decorator and util functions

• feat(#11): timeout limit context manager.

• refactor(core): several minor refactors

• fix(docs): comment out latex sphinx settings

• fix(docs): use current package version

• Documentation examples improvements (#27)

• feat(history): update

• feat: add pool length magic method

0.1.5 (2016-12-04)

• fix(#25): allow empty iterables in iterators functions, such as map, filter, reduce.

0.1.4 (2016-11-28)

• fix(#24): explicitly pass loop instance to asyncio.wait.

0.1.3 (2016-10-27)

• feat(#17): add flat_map function.

• feat(#18): add pipeline-style operator overloading composition.

0.1.2 (2016-10-25)

• fix(setup.py): fix pip installation.

• refactor(api): minor refactors in several functions and tests.

0.1.1 (2016-10-24)

• refactor(name): use new project name.

0.1.0 (2016-10-23)

• First version (beta)