Skip to main content

更简单的使用异步协程和线程池

Project description

newpool

更简单的使用异步协程和线程池

安装

pip install newpool

使用

异步协程

newpool.async_gather() 并发执行多个协程, 返回结果列表, 遇到异常不会中断, 而是将错误原因返回到结果列表中 newpool.async_taskgroup() 并发执行多个协程, 返回列表, 包含更详细的状态, 遇到异常会中断

import newpool

async def test():
    await newpool.async_sleep(1)
    print("test")

gather_results = newpool.async_gather([test() for i in range(20)], sem=5) # sem 为并发协程数
taskgroup_results = newpool.async_taskgroup([test() for i in range(20)]) # 此处没有使用 sem, 默认基于设备自动计算并发协程数

线程池

基于 concurrent.futures.ThreadPoolExecutor 实现

import newpool
import time

pool = newpool.Pool(5)

@pool.task
def test():
    time.sleep(2)
    print("test")

for i in range(20):
    test()

pool.wait() # 等待所有任务执行完毕
pool.state() # 线程池状态
pool.results # 所有任务执行结果
pool.cancel_all() # 取消所有任务

内置可以直接使用的线程池 newpool.pool, 无需实例化。 线程数默认基于设备自动计算, 也可以通过 newpool.pool.n = 5 修改线程数

import newpool

@newpool.pool.task
def test():
    print("test")

for i in range(20):
    test()

newpool.pool.wait() # 等待所有任务执行完毕
newpool.pool.state() # 线程池状态
newpool.pool.results # 所有任务执行结果
newpool.pool.cancel_all() # 取消所有任务

基于 asyncio 实现的线程池

需要通过 asyncio 运行, 可以直接使用 newpool.async_gather() 或者 newpool.async_taskgroup()

默认线程数基于设备自动计算,也可以通过 newpool.to_async(n=5) 修改线程数。

import newpool
import time

@newpool.to_async()
def test():
    time.sleep(2)
    print("test")

newpool.async_gather([test() for i in range(20)])

@newpool.to_async(n=5)
def test2():
    time.sleep(2)
    print("test")

newpool.async_gather([test2() for i in range(20)])

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

newpool-0.0.2.tar.gz (4.2 kB view details)

Uploaded Source

Built Distribution

newpool-0.0.2-py3-none-any.whl (4.5 kB view details)

Uploaded Python 3

File details

Details for the file newpool-0.0.2.tar.gz.

File metadata

  • Download URL: newpool-0.0.2.tar.gz
  • Upload date:
  • Size: 4.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: pdm/2.17.1 CPython/3.11.3 Windows/10

File hashes

Hashes for newpool-0.0.2.tar.gz
Algorithm Hash digest
SHA256 66c57e00b0141f290d22895cd1a36de149ee4e62dd071ed9f431904737b3d9b6
MD5 e5bdce5e15563a69ea00d79d5803294b
BLAKE2b-256 5d30145dd243e34e1bf0fc3100432ffdb5939ec5d55016d37871768ed725d5db

See more details on using hashes here.

File details

Details for the file newpool-0.0.2-py3-none-any.whl.

File metadata

  • Download URL: newpool-0.0.2-py3-none-any.whl
  • Upload date:
  • Size: 4.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: pdm/2.17.1 CPython/3.11.3 Windows/10

File hashes

Hashes for newpool-0.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 e58ad2334d4ede5eb01d6709591b5e835fdc01be143447a55ba4bd1e5caec0e4
MD5 27df7608605042bc4f87bd6947bc675a
BLAKE2b-256 011e5b9668e58ce33bd263e7f1d7ba56e803d6956309cccc012c5d7a3f1ae3d7

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page