Non-blocking Python methods using decorators
MultiTasking: Non-blocking Python methods using decorators
MultiTasking is a tiny Python library lets you convert your Python methods into asynchronous, non-blocking methods simply by using a decorator.
# example.py import multitasking import time import random import signal # kill all tasks on ctrl-c signal.signal(signal.SIGINT, multitasking.killall) # or, wait for task to finish on ctrl-c: # signal.signal(signal.SIGINT, multitasking.wait_for_tasks) @multitasking.task # <== this is all it takes :-) def hello(count): sleep = random.randint(1,10)/2 print("Hello %s (sleeping for %ss)" % (count, sleep)) time.sleep(sleep) print("Goodbye %s (after for %ss)" % (count, sleep)) if __name__ == "__main__": for i in range(0, 10): hello(i+1)
The output would look something like this:
$ python example.py Hello 1 (sleeping for 0.5s) Hello 2 (sleeping for 1.0s) Hello 3 (sleeping for 5.0s) Hello 4 (sleeping for 0.5s) Hello 5 (sleeping for 2.5s) Hello 6 (sleeping for 3.0s) Hello 7 (sleeping for 0.5s) Hello 8 (sleeping for 4.0s) Hello 9 (sleeping for 3.0s) Hello 10 (sleeping for 1.0s) Goodbye 1 (after for 0.5s) Goodbye 4 (after for 0.5s) Goodbye 7 (after for 0.5s) Goodbye 2 (after for 1.0s) Goodbye 10 (after for 1.0s) Goodbye 5 (after for 2.5s) Goodbye 6 (after for 3.0s) Goodbye 9 (after for 3.0s) Goodbye 8 (after for 4.0s) Goodbye 3 (after for 5.0s)
The default maximum threads is equal to the # of CPU Cores. This is just a rule of thumb! The Thread module isn’t actually using more than one core at a time.
You can change the default maximum number of threads using:
import multitasking multitasking.set_max_threads(10)
…or, if you want to set the maximum number of threads based on the number of CPU Cores, you can:
import multitasking multitasking.set_max_threads(multitasking.config["CPU_CORES"] * 5)
For applications that doesn’t require access to shared resources, you can set MultiTasking to use multiprocessing.Process() instead of the threading.Thread(), thus avoiding some of the GIL constraints.
import multitasking multitasking.set_engine("process") # "process" or "thread"
Install multitasking using pip:
$ pip install multitasking --upgrade --no-cache-dir
MultiTasking is distributed under the GNU Lesser General Public License v3.0. See the LICENSE.txt file in the release for details. MultiTasking is not a product of Interactive Brokers, nor is it affiliated with Interactive Brokers.