A javascript style promises for python that uses multicore and threads combined. Go Fast
Project description
# python-promises
Promises for Python
This project was made to merge Multic-core processing and threading in python via a node Javascript style promises package. Having simple power over GIL and multi-core with the beauty of Promises is nice
## Examples
```python
def somefunction (args):
print 'hi'
def crunchnumber ( args ):
pi = 3.14
for x in range ( 0, 1000000):
pi = pi * pi
# Could use args and compute neural nets or cpu intensive work - this was called multicore
return pi
# Simple case
p = Promise(somefunction).then (rejected=myerror)
# Executed with no waiting. myerror is called if there was a problem
# Wait Case
p = Promise (somefunction).then (resolved=mysuccess)
if p.wait():
print 'all completed successfully'
# Mixing threads and Multiprocessing ones
for x in range (0,20):
p.append ( Promise (crunchnumber, somemodel, somestart).then ( multicore=True, resovled=get_computation_result)
for x in range (0,10:
p.append ( Promise (someiofunction).then ()
# Wait for all threads and multi-core promises to finish
Promise.wait_all ( p )
```
## Why ?
For Python 2.7+. Node has such a wonderful way to handle server side. With all of the debates
on what to do with Python around GIL, I decided to support both GIL via threads and also multicore in the same
API structure. We are doing data science work and need multi-core there and threads for I/O
Threads version (default) uses GIL and therefore all your data is thread safe. It just runs.
Multicore uses multiprocessing. Effectively, the script is forked to another core and the process is monitored.
When the function finishes, one can return data but only if it is JSON seralizable.
Notes: For I/O blocking requests, threads keeps it simple as it uses GIL and you don't have to worry about thread locking or safety. However, for true multicore processing, you can set multicore=true and bypass GIL. This is fantastic for hard computing tasks and when you need the computing power of the machine. Same syntax, just a flag.
Promises for Python
This project was made to merge Multic-core processing and threading in python via a node Javascript style promises package. Having simple power over GIL and multi-core with the beauty of Promises is nice
## Examples
```python
def somefunction (args):
print 'hi'
def crunchnumber ( args ):
pi = 3.14
for x in range ( 0, 1000000):
pi = pi * pi
# Could use args and compute neural nets or cpu intensive work - this was called multicore
return pi
# Simple case
p = Promise(somefunction).then (rejected=myerror)
# Executed with no waiting. myerror is called if there was a problem
# Wait Case
p = Promise (somefunction).then (resolved=mysuccess)
if p.wait():
print 'all completed successfully'
# Mixing threads and Multiprocessing ones
for x in range (0,20):
p.append ( Promise (crunchnumber, somemodel, somestart).then ( multicore=True, resovled=get_computation_result)
for x in range (0,10:
p.append ( Promise (someiofunction).then ()
# Wait for all threads and multi-core promises to finish
Promise.wait_all ( p )
```
## Why ?
For Python 2.7+. Node has such a wonderful way to handle server side. With all of the debates
on what to do with Python around GIL, I decided to support both GIL via threads and also multicore in the same
API structure. We are doing data science work and need multi-core there and threads for I/O
Threads version (default) uses GIL and therefore all your data is thread safe. It just runs.
Multicore uses multiprocessing. Effectively, the script is forked to another core and the process is monitored.
When the function finishes, one can return data but only if it is JSON seralizable.
Notes: For I/O blocking requests, threads keeps it simple as it uses GIL and you don't have to worry about thread locking or safety. However, for true multicore processing, you can set multicore=true and bypass GIL. This is fantastic for hard computing tasks and when you need the computing power of the machine. Same syntax, just a flag.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
gofast-0.1.tar.gz
(3.2 kB
view hashes)