Pool executor supporting thread, process and async.
Project description
HybridPoolExecutor
HybridPoolExecutor is a parallel executor that supports thead, process and async three operating models at the same time.
Example:
import asyncio
import time
import random
from hybrid_pool_executor import HybridPoolExecutor
def solve(v: int) -> int:
print(f"You give worker a value {v}")
time.sleep(random.randint(1, 4))
print(f"The square of value {v} is {v * v}")
return v
async def solve_async(v: int) -> int:
print(f"You give async worker a value {v}")
await asyncio.sleep(random.randint(1, 4))
print(f"The square of value {v} is {v * v}")
return v
async def main():
pool = HybridPoolExecutor()
# run in a thread
future0 = pool.submit(solve, 0)
# run in a process by setting "_mode" to "process"
future1 = pool.submit(solve, 1, _mode="process")
# or you can use a more precise approach: submit_task
future2 = pool.submit_task(solve, kwargs={"v": 2}, mode="process")
# run in an async worker
future3 = pool.submit(solve_async, 3) # run as a coroutine
# you can also submit a coroutine
future4 = pool.submit(solve_async(4))
# async function/coroutine can be executed in thread/process too
# run coroutine in a thread
future5 = pool.submit(solve_async(5), _mode="thread")
# or in a process
future6 = pool.submit(solve_async, 6, _mode="process")
# all futures can be get either synchronously or asynchronously
await future0 # result from a thread worker
await future1 # result from a process worker
future2.result() # result from a async worker
# ......
if __name__ == "__main__":
asyncio.run(main())
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
hybrid-pool-executor-0.0.4.tar.gz
(29.7 kB
view hashes)
Built Distribution
Close
Hashes for hybrid-pool-executor-0.0.4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | fd8a6fbe3853fb6cdcce643b2c37ed615c27cb418d803d966edd803b4e2d74fe |
|
MD5 | 78441ebc948d3f8370a6476a66ec31d3 |
|
BLAKE2b-256 | 0a70469854956fb0b4c19a3f63a1f7a234a935f1c455ead92fad1f0c6e44426a |
Close
Hashes for hybrid_pool_executor-0.0.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c7dbd9aabcaeb7f38f3e0147428e167cddca7c7ed8c73083b15256f5d5fb39e4 |
|
MD5 | f808bd9f35064b15a5f7949e1db66e94 |
|
BLAKE2b-256 | 62cfb048a3f60d24f4f2ff0c75d5a58e10452c01dfc07f2194cd1c50b736b0c4 |