Skip to main content

AsyncIO, threading and multiprocessing combined to created a powerful executor

Project description

aiomultithreading:

PyPI version PyPI - Downloads License: MIT Tests pre-commit Checked with mypy Code style: black Imports: isort

What is it?

aiomultithreading is

A concept for creating the Best Possible Utilization of threading, multiprocessing and asyncio combined into one executor allowing the number of tasks running in parallel to be multiplied rapidly without having to do very many setups.

Aiomultithreading is meant for handling super bulky tasks such as proxy-enumeration or networking on a very large scale. This can be costly for some uneducated programmers but luckily this tool completely changes that by making use of not just your threads, cores or task counts alone, No, All of them combined. This library completely changes the playing-field and I expect to see a few people using this for something really intense.

With the combined help of aiomultiprocessing and aiothreading it is possible to acheieve the maximum amount of tasks that can be quickly ran in parallel.

Even the default tasks per child with threading and cores will surprise you.

Even with the use of a low cpu machine when doing the math (2 processes * 4 threads * 16 tasks per thread) it makes a total of 128 tasks. Theses numbers can rapidly multiply when being used pc or device using a higher cpu count.

This library also has compatability for winloop and uvloop right out of the box.

FYI

  • If your goals involved some form of brute-forcing that's offline, please note that Asyncio is not the best canidate for this. Use hashcat if your trying to grind those kinds of things out.

Usage:

Dependencies

Installing

The easiest way is to install aiomultithreading is from PyPI using pip:

pip install aiomultithreading

Running

First, import the library.

from aiomultithreading import MultiPool
import asyncio 

async def sleep(i:float):
    await asyncio.sleep(i)
    return f"slept for {i}"

async def main():
    # 2 processes and 6 threads...
    async with MultiPool(2, 6) as pool:
        # You can go extremely fast with even 500 of these stacked together...
        x = [i for i in range(5)] * 100
        print(x)
        async for text in pool.map(sleep, x):
            print(text)

if __name__ == "__main__":
    asyncio.run(main())

TODO:

- Optimize some more important parts

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

aiomultithreading-0.1.1-py3-none-any.whl (10.7 kB view details)

Uploaded Python 3

File details

Details for the file aiomultithreading-0.1.1-py3-none-any.whl.

File metadata

File hashes

Hashes for aiomultithreading-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 f8b40b228ecad0763500a2b25c3f01b69ebf479592592a13702cfc0e3743779d
MD5 c69da04d5d6a41dbea0d5aaf0ec76212
BLAKE2b-256 dd1da38475284489af70f8ec15363bd72abba624aa193df8a2c417d1ffbba4fc

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page