Non-blocking Python methods using decorators
MultiTasking: Non-blocking Python methods using decorators
MultiTasking is a tiny Python library lets you convert your Python methods into asynchronous, non-blocking methods simply by using a decorator.
# example.py import multitasking import time import random import signal # kill all tasks on ctrl-c signal.signal(signal.SIGINT, multitasking.killall) # or, wait for task to finish on ctrl-c: # signal.signal(signal.SIGINT, multitasking.wait_for_tasks) @multitasking.task # <== this is all it takes :-) def hello(count): sleep = random.randint(1,10)/2 print("Hello %s (sleeping for %ss)" % (count, sleep)) time.sleep(sleep) print("Goodbye %s (after for %ss)" % (count, sleep)) if __name__ == "__main__": for i in range(0, 10): hello(i+1)
The output would look something like this:
$ python example.py Hello 1 (sleeping for 0.5s) Hello 2 (sleeping for 1.0s) Hello 3 (sleeping for 5.0s) Hello 4 (sleeping for 0.5s) Hello 5 (sleeping for 2.5s) Hello 6 (sleeping for 3.0s) Hello 7 (sleeping for 0.5s) Hello 8 (sleeping for 4.0s) Hello 9 (sleeping for 3.0s) Hello 10 (sleeping for 1.0s) Goodbye 1 (after for 0.5s) Goodbye 4 (after for 0.5s) Goodbye 7 (after for 0.5s) Goodbye 2 (after for 1.0s) Goodbye 10 (after for 1.0s) Goodbye 5 (after for 2.5s) Goodbye 6 (after for 3.0s) Goodbye 9 (after for 3.0s) Goodbye 8 (after for 4.0s) Goodbye 3 (after for 5.0s)
The default maximum threads is equal to the # of CPU Cores. This is just a rule of thumb! The Thread module isn’t actually using more than one core at a time.
You can change the default maximum number of threads using:
import multitasking multitasking.set_max_threads(10)
…or, if you want to set the maximum number of threads based on the number of CPU Cores, you can:
import multitasking multitasking.set_max_threads(multitasking.config["CPU_CORES"] * 5)
For applications that doesn’t require access to shared resources, you can set MultiTasking to use multiprocessing.Process() instead of the threading.Thread(), thus avoiding some of the GIL constraints.
import multitasking multitasking.set_engine("process") # "process" or "thread"
Install multitasking using pip:
$ pip install multitasking --upgrade --no-cache-dir
Install multitasking using conda:
$ conda install -c ranaroussi multitasking
MultiTasking is distributed under the Apache Software License. See the LICENSE.txt file in the release for details.
Please drop me an note with any feedback you have.
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.