submitting cpu-bound tasks to processes and io-bound tasks to threads
Convert a classic sequential program into a parallel one.
It runs faster.
What if not?
Don’t use it.
for image in images: create_thumbnail(image) # original for image in images: fork(create_thumbnail, image) # parallelized explicitly for image in images: create_thumbnail(image) # parallelized implicitly (read below)
What about return values?
result = fork(my_func, *args, **kwargs)
That is a proxy object that behaves almost exactly as if it were the real return value of my_func: Furthermore, result proxies evaluate lazily, e.g. operators (like +, - etc.) evaluate until really needed.
What about exceptions?
In case of an error, you will receive the normal traceback that you would see in the sequential case.
Speaking of threads …
and processes? fork will take care of that for you.
You can assist fork by decorating your functions (not decorating defaults to cpu_bound):
@io_bound def call_remote_webservice(): # implementation @cpu_bound def fib(n): # naive implementation of Fibonacci numbers @unsafe # don't fork; run sequentially def weird_side_effects(*args, **kwargs): # implementation
If you don’t like the fork calling syntax, you can convert certain functions into forks.
Use with caution.
@io_bound_fork def create_thumbnail_by_webservice(image): # implementation @cpu_bound_fork def create_thumbnail_by_bare_processing_power(image): # implementation # the following two lines spawn two forks create_thumbnail_by_webservice(image1) create_thumbnail_by_bare_processing_power(image2)
- easy way back and forth (from sequential to parallel and vice versa)
- results evaluate lazily
- tracebacks are preserved
- cascading possible (thread-safe)
- compatible with Python 2 and 3
- weird calling syntax (no syntax support)
- type(result) == ResultProxy
- not working with lambdas due to PickleError
- needs fix:
- “maximum recursion depth exceeded” due to encapsulating all operations into proxies
- not working with coroutines (asyncio) yet
Release history Release notifications
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size xfork-0.20-py2-none-any.whl (4.2 kB)||File type Wheel||Python version py2||Upload date||Hashes View hashes|
|Filename, size xfork-0.20-py3-none-any.whl (4.1 kB)||File type Wheel||Python version py3||Upload date||Hashes View hashes|
|Filename, size xfork-0.20.tar.gz (4.8 kB)||File type Source||Python version None||Upload date||Hashes View hashes|