Skip to main content

MultiTasking: Non-blocking Python methods using decorators

Python version Travis-CI build status PyPi version PyPi status PyPi downloads CodeFactor Star this repo Follow me on twitter

MultiTasking is a tiny Python library lets you convert your Python methods into asynchronous, non-blocking methods simply by using a decorator.

Example

# example.py
import multitasking
import time
import random
import signal

# kill all tasks on ctrl-c
signal.signal(signal.SIGINT, multitasking.killall)

# or, wait for task to finish on ctrl-c:
# signal.signal(signal.SIGINT, multitasking.wait_for_tasks)

@multitasking.task # <== this is all it takes :-)
def hello(count):
    sleep = random.randint(1,10)/2
    print("Hello %s (sleeping for %ss)" % (count, sleep))
    time.sleep(sleep)
    print("Goodbye %s (after for %ss)" % (count, sleep))

if __name__ == "__main__":
    for i in range(0, 10):
        hello(i+1)

The output would look something like this:

$ python example.py

Hello 1 (sleeping for 0.5s)
Hello 2 (sleeping for 1.0s)
Hello 3 (sleeping for 5.0s)
Hello 4 (sleeping for 0.5s)
Hello 5 (sleeping for 2.5s)
Hello 6 (sleeping for 3.0s)
Hello 7 (sleeping for 0.5s)
Hello 8 (sleeping for 4.0s)
Hello 9 (sleeping for 3.0s)
Hello 10 (sleeping for 1.0s)
Goodbye 1 (after for 0.5s)
Goodbye 4 (after for 0.5s)
Goodbye 7 (after for 0.5s)
Goodbye 2 (after for 1.0s)
Goodbye 10 (after for 1.0s)
Goodbye 5 (after for 2.5s)
Goodbye 6 (after for 3.0s)
Goodbye 9 (after for 3.0s)
Goodbye 8 (after for 4.0s)
Goodbye 3 (after for 5.0s)

Settings

The default maximum threads is equal to the # of CPU Cores. This is just a rule of thumb! The Thread module isn’t actually using more than one core at a time.

You can change the default maximum number of threads using:

import multitasking
multitasking.set_max_threads(10)

…or, if you want to set the maximum number of threads based on the number of CPU Cores, you can:

import multitasking
multitasking.set_max_threads(multitasking.config["CPU_CORES"] * 5)

For applications that doesn’t require access to shared resources, you can set MultiTasking to use multiprocessing.Process() instead of the threading.Thread(), thus avoiding some of the GIL constraints.

import multitasking
multitasking.set_engine("process") # "process" or "thread"

Installation

Install multitasking using pip:

$ pip install multitasking --upgrade --no-cache-dir

Install multitasking using conda:

$ conda install -c ranaroussi multitasking

Metadata

Release files for python-multitasking 0.0.10

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for python-multitasking 0.0.10
File Size Uploaded
python-multitasking-0.0.10.tar.gz 8.4 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for python-multitasking 0.0.10
File Interpreter ABI Platform
python_multitasking-0.0.10-py3-none-any.whl Python 3 none any Details

Total release size: 17.0 kB

Release files / python-multitasking-0.0.10.tar.gz

Download URL python-multitasking-0.0.10.tar.gz
Size 8.4 kB
Tags Source
SHA-256 checksum
How to use checksums
6b74a10b810f12936cfcd4c0a2eef068573ab8339d1abef9e47dc7194550f6cb
BLAKE2b-256 checksum
How to use checksums
992f2235dcb79a3a0d60d71835931ae1d3578854c1aa5ff8fcb77fb6169a2a36
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.2.0 pkginfo/1.6.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.52.0 CPython/3.8.5

Release files / python_multitasking-0.0.10-py3-none-any.whl

Download URL python_multitasking-0.0.10-py3-none-any.whl
Size 8.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
eb844c1397e7231f3b7e02222b0efa65f22553e3fb8743ab3a2275f99e1f6c4b
BLAKE2b-256 checksum
How to use checksums
5df56e79f0096136f4bb6291d93682f3bf5fe1ad558ac6067987806a2d2eb935
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.2.0 pkginfo/1.6.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.52.0 CPython/3.8.5

Release history Release notifications | RSS feed

This release

0.0.10 This release

2 release files

0.0.9

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page