Convert a classic sequential program into a parallel one.
Why?
It runs faster.
What if not?
Don’t use it.
How?
for image in images:
create_thumbnail(image) # original
for image in images:
fork(create_thumbnail, image) # parallelized explicitly
for image in images:
create_thumbnail(image) # parallelized implicitly (read below)What about return values?
result = fork(my_func, *args, **kwargs)And what is this result?
A future that behaves almost exactly as if it were the return value of my_func. That in turn means, as soon as you access the result and it is not ready yet, the main thread blocks.
Speaking of threads …
and processes? fork will take care of that for you.
You can assist fork by decorating your functions (not decorating defaults to cpu_bound):
@io_bound
def call_remote_webservice():
# implementation
@cpu_bound
def fib(n):
# naive implementation of Fibonacci numbers
@unsafe # don't fork; run sequentially
def weird_side_effects(*args, **kwargs):
# implementationParallelize implicitly?
If you don’t like the fork calling syntax, you can convert certain functions into forks.
Use with caution.
@io_bound_fork
def create_thumbnail_by_webservice(image):
# implementation
@cpu_bound_fork
def create_thumbnail_by_bare_processing_power(image):
# implementation
# the following two lines spawn two forks
create_thumbnail_by_webservice(image1)
create_thumbnail_by_bare_processing_power(image2)I still need more performance.
You feel like debugging is still too easy, don’t you? Go ahead with contagious futures.
Use with extreme caution.
NOTE: decorator ‘contagious’ was renamed to ‘contagious_result’.
@io_bound
def item():
# implementation
result = 0
for item in items:
result += fork_contagious(item) # explicit
print(result)
# or
@io_bound # also works with fork decorator
@contagious_result
def item():
# implementation
result = 0
for item in items:
result += fork(item) # implicit
print(result)Conclusion
Good
easy way back and forth (from sequential to parallel and vice versa)
cascading possible (thread-safe)
compatible with Python 2 and 3
Bad
weird calling syntax (no syntax support)
type(result) == FutureWrapper
not working with lambdas due to PickleError
needs fix:
contagious_result (caller has no control over result; might get deprecated)
still needs some mechanism to wait and to evaluate all contagious results (using ‘with’ or function call)
not working with coroutines (asyncio) yet
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file xfork-0.16.tar.gz.
File metadata
- Download URL: xfork-0.16.tar.gz
- Upload date:
- Size: 5.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a0c3a25380fd4e44c194a24a24560fb9fb5ec57ceed9b1bf654f533377c2d16f
|
|
| MD5 |
4a371b2a7248269026d3f17635113b5b
|
|
| BLAKE2b-256 |
a48a0c86b1e5c2ef2b558ec3b8ab84aa70d2531ab3531f83439ba7635f82cdab
|
File details
Details for the file xfork-0.16-py3-none-any.whl.
File metadata
- Download URL: xfork-0.16-py3-none-any.whl
- Upload date:
- Size: 4.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e1d576bea1712e62ca784f8804606df8ed671f92d5a03a55630082224bd99f47
|
|
| MD5 |
20a3ec1c07d13ce04809c5d0c9f9fd1a
|
|
| BLAKE2b-256 |
2e1c2ecc1a52afcc9ebe2cc97a6d16bc0dcd1d70465fd8ac600601ccba1f9007
|
File details
Details for the file xfork-0.16-py2-none-any.whl.
File metadata
- Download URL: xfork-0.16-py2-none-any.whl
- Upload date:
- Size: 4.3 kB
- Tags: Python 2
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
eb31be9e9e80e0e63ec975f2e7e1c4afd8cb960e7a84b6b971f065d9bc5d19a1
|
|
| MD5 |
21d3b4f34a4f04c690495cb1da0e1fb8
|
|
| BLAKE2b-256 |
ef51a4bdef9f1e929337456e450f47c9a9bd1f9d54ff34253a5507e50a55e589
|
