Convert a classic sequential program into a parallel one.
Why?
It runs faster.
What if not?
Don’t use it.
How?
for image in images:
create_thumbnail(image) # original
for image in images:
fork(create_thumbnail, image) # parallelized explicitly
for image in images:
create_thumbnail(image) # parallelized implicitly (read below)What about return values?
result = fork(my_func, *args, **kwargs)And what is this result?
A future that behaves almost exactly as if it were the return value of my_func. That in turn means, as soon as you access the result and it is not ready yet, the main thread blocks.
Speaking of threads …
and processes? fork will take care of that for you.
You can assist fork by decorating your functions (not decorating defaults to cpu_bound):
@io_bound
def call_remote_webservice():
# implementation
@cpu_bound
def fib(n):
# naive implementation of Fibonacci numbers
@unsafe # don't fork; run sequentially
def weird_side_effects(*args, **kwargs):
# implementationParallelize implicitly?
If you don’t like the fork calling syntax, you can convert certain functions into forks.
Use with caution.
@io_bound_fork
def create_thumbnail_by_webservice(image):
# implementation
@cpu_bound_fork
def create_thumbnail_by_bare_processing_power(image):
# implementation
# the following two lines spawn two forks
create_thumbnail_by_webservice(image1)
create_thumbnail_by_bare_processing_power(image2)I still need more performance.
You feel like debugging is still too easy, don’t you? Go ahead with contagious futures.
Use with extreme caution.
NOTE: decorator ‘contagious’ was renamed to ‘contagious_result’.
@io_bound
def item():
# implementation
result = 0
for item in items:
result += fork_contagious(item) # explicit
print(result)
# or
@io_bound # also works with fork decorator
@contagious_result
def item():
# implementation
result = 0
for item in items:
result += fork(item) # implicit
print(result)Conclusion
Good
easy way back and forth (from sequential to parallel and vice versa)
cascading possible (thread-safe)
compatible with Python 2 and 3
Bad
weird calling syntax (no syntax support)
type(result) == FutureWrapper
not working with lambdas due to PickleError
needs fix:
contagious_result (caller has no control over result; might get deprecated)
still needs some mechanism to wait and to evaluate all contagious results (using ‘with’ or function call)
not working with coroutines (asyncio) yet
1 maintainer
Release files for xfork 0.16
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| xfork-0.16.tar.gz | 5.1 kB | Details |
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| xfork-0.16-py3-none-any.whl | Python 3 | none | any | Details |
| xfork-0.16-py2-none-any.whl | Python 2 | none | any | Details |
Total release size:13.7 kB
Release files / xfork-0.16.tar.gz
| Download URL | xfork-0.16.tar.gz |
|---|---|
| Size | 5.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a0c3a25380fd4e44c194a24a24560fb9fb5ec57ceed9b1bf654f533377c2d16f
|
|
BLAKE2b-256 checksum How to use checksums |
a48a0c86b1e5c2ef2b558ec3b8ab84aa70d2531ab3531f83439ba7635f82cdab
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Release files / xfork-0.16-py3-none-any.whl
| Download URL | xfork-0.16-py3-none-any.whl |
|---|---|
| Size | 4.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
e1d576bea1712e62ca784f8804606df8ed671f92d5a03a55630082224bd99f47
|
|
BLAKE2b-256 checksum How to use checksums |
2e1c2ecc1a52afcc9ebe2cc97a6d16bc0dcd1d70465fd8ac600601ccba1f9007
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Release files / xfork-0.16-py2-none-any.whl
| Download URL | xfork-0.16-py2-none-any.whl |
|---|---|
| Size | 4.3 kB |
| Tags | Python 2 |
|
SHA-256 checksum How to use checksums |
eb31be9e9e80e0e63ec975f2e7e1c4afd8cb960e7a84b6b971f065d9bc5d19a1
|
|
BLAKE2b-256 checksum How to use checksums |
ef51a4bdef9f1e929337456e450f47c9a9bd1f9d54ff34253a5507e50a55e589
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
