Pool executor supporting thread, process and async.
Project description
HybridPoolExecutor
HybridPoolExecutor is a parallel executor that supports thead, process and async three operating models at the same time.
Example:
import asyncio
import time
import random
from hybrid_pool_executor import HybridPoolExecutor
def solve(v: int) -> int:
print(f"You give worker a value {v}")
time.sleep(random.randint(1, 4))
print(f"The square of value {v} is {v * v}")
return v
async def solve_async(v: int) -> int:
print(f"You give async worker a value {v}")
await asyncio.sleep(random.randint(1, 4))
print(f"The square of value {v} is {v * v}")
return v
async def main():
pool = HybridPoolExecutor()
# run in a thread
future0 = pool.submit(solve, 0)
# run in a process by setting "_mode" to "process"
future1 = pool.submit(solve, 1, _mode="process")
# or you can use a more precise approach: submit_task
future2 = pool.submit_task(solve, kwargs={"v": 2}, mode="process")
# run in an async worker
future3 = pool.submit(solve_async, 3) # run as a coroutine
# you can also submit a coroutine
future4 = pool.submit(solve_async(4))
# async function/coroutine can be executed in thread/process too
# run coroutine in a thread
future5 = pool.submit(solve_async(5), _mode="thread")
# or in a process
future6 = pool.submit(solve_async, 6, _mode="process")
# all futures can be get either synchronously or asynchronously
await future0 # result from a thread worker
await future1 # result from a process worker
future2.result() # result from a async worker
# ......
if __name__ == "__main__":
asyncio.run(main())
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
hybrid-pool-executor-0.0.3.tar.gz
(30.1 kB
view hashes)
Built Distribution
Close
Hashes for hybrid-pool-executor-0.0.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0b71fc83bd2322db5a54459e4cd7090bfa26096751f540939491062ea7cd498b |
|
MD5 | e6949182803451eeadcc7774bf08905a |
|
BLAKE2b-256 | 1d5bb041349a448ca59c63b7c074a53ead372e4c0b57ab9aaaa8f4eef968c9b3 |
Close
Hashes for hybrid_pool_executor-0.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 17529b1d9bc8bd56a112995d7c67024df9e889449572815f369a216930b8b57f |
|
MD5 | 075b035a750411b84c262d50a5abbe43 |
|
BLAKE2b-256 | 5e144da36f34541eb3d517f361f68887ff1274de81ab9ad57ce2266e8d0703ae |