Skip to main content

CoroutinePoolExecutor compatible to ThreadPoolExecutor and ProcessPoolExecutor

Project description

Though CoExecutor uses 1 CPU core at any time,

limitations such as ‘max_workers’ are needed for limiting concurrent asyncio connections.

This is useful when you are building a asyncio-based web crawler.

It is designed to have exactly same result with respect to python’s ThreadPoolExecutor and ProcessPoolExecutor.

Also, CoExecutor supports additional features like ‘limit’ or ‘out_of_order’.…, limit=X) limits the number of pending futures (done, but not collected yet).…, out_of_order=True) ignores the input iterator’s ordering.

The results are collected by the termination order of the jobs.

CoExecutor constructor requires a loop to rely on.

With debug=True, it will print the number of active coroutines per second.

Works with Python3.5+.

Project details

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page