Write a classic sequential program. Then convert it into a parallel one.
Why?
It runs faster.
What if not?
Don’t use it.
How?
for image in images:
create_thumbnail(image) # original
for image in images:
fork(create_thumbnail, image) # parallelized explicitly
for image in images:
create_thumbnail(image) # parallelized implicitly (read below)What about return values?
As usual:
result = fork(my_func, *args, **kwargs)It’s a proxy object that behaves almost exactly like the real return value of my_func. Furthermore, it evaluates only if needed; also in combination with operators (like +, - etc.).
What happens to exceptions?
Their original (sequential) tracebacks are preserved. That should make debugging easier. However, don’t try to catch exceptions. You better want to exit and see them.
Speaking of threads …
and processes? fork will take care of that for you.
You can assist fork by decorating your functions (not decorating defaults to cpu_bound):
@io_bound
def call_remote_webservice():
# implementation
@cpu_bound
def fib(n):
# naive implementation of Fibonacci numbers
@unsafe # don't fork; run sequentially
def weird_side_effects(*args, **kwargs):
# implementationParallelize implicitly?
If you don’t like the fork calling syntax, you can convert certain functions into stand-alone forks.
Use with caution.
@io_bound_fork
def create_thumbnail_by_webservice(image):
# implementation
@cpu_bound_fork
def create_thumbnail_by_bare_processing_power(image):
# implementation
# the following two lines spawn two forks
create_thumbnail_by_webservice(image1)
create_thumbnail_by_bare_processing_power(image2)Conclusion
Good
easy to give it a try / easy way from sequential to parallel and back
results evaluate lazily
sequential tracebacks are preserved
it’s thread-safe / cascading forks possible
compatible with Python 2 and 3
Bad
weird calling syntax (no syntax support)
type(result) == ResultProxy
not working with lambdas due to PickleError
needs fix:
fix exception handling somehow
not working with coroutines (asyncio) yet
1 maintainer
Release files for xfork 0.25
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| xfork-0.25.tar.gz | 5.1 kB | Details |
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| xfork-0.25-py3-none-any.whl | Python 3 | none | any | Details |
| xfork-0.25-py2-none-any.whl | Python 2 | none | any | Details |
Total release size: 13.9 kB
Release files / xfork-0.25.tar.gz
| Download URL | xfork-0.25.tar.gz |
|---|---|
| Size | 5.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
bdc6006bc023bfb8f8a8eb0d15d57788cdc2dec5694e59d587b3d7721dd54527
|
|
BLAKE2b-256 checksum How to use checksums |
219bfda6e98e2bcf5f9e0da4ea380cadaa3af625404afb32292c355d77e1e53d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Release files / xfork-0.25-py3-none-any.whl
| Download URL | xfork-0.25-py3-none-any.whl |
|---|---|
| Size | 4.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9f8f821997a4fd3fdeddb00ae517aad4b92ffe0e579a80565a1a46691a74d0a6
|
|
BLAKE2b-256 checksum How to use checksums |
f51a10639c3b24370b6dc9031c2747f30d1a708fef95c6b49d519f2485670cb2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Release files / xfork-0.25-py2-none-any.whl
| Download URL | xfork-0.25-py2-none-any.whl |
|---|---|
| Size | 4.4 kB |
| Tags | Python 2 |
|
SHA-256 checksum How to use checksums |
1961272ab666dc833119a19366b6345bd4b5b522357735e2cd750a050443ba58
|
|
BLAKE2b-256 checksum How to use checksums |
5b45104f6d7e1dc51e594a5ea1c269cdb6e10ff9ee180a23b4f8fed8311591c9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
