KnKPoolExecutor 1.0.2

Extension of concurrent.futures for Cloud-Function-as-a-Service compatibility

Kanokkul's Pool Executor

Kanokkul's Pool Executor contains ProcessPoolExecutor and ThreadPoolExecutor which preserve traceback output for exception. This package also has StackTracedProcessPoolExecutor for environment without /dev/shm

Installation

pip install KnKPoolExecutor

Usage

Simply substitute ProcessPoolExecutor with PipeProcessPoolExecutor or StackTracedProcessPoolExecutor
Replace ThreadPoolExecutor with StackTracedThreadPoolExecutor

from knk_pool_executor import PipeProcessPoolExecutor

with PipeProcessPoolExecutor() as pool:
    future = pool.submit(some_function, arg, kw=kw)
    
    # do another thing

    result = future.result()

Inspiration

• Parallel Processing in Python with AWS Lambda: AWS Compute Blog
• How to emulate multiprocessing.Pool.map() in AWS Lambda?
• se7entyse7en's answer in StackOverflow about Getting original line number for exception in concurrent.futures