===
__tasks.py__ is a simple and fast task queue for executing multiple tasks in parallel. All you need to do is specify the task as a simple function that takes an argument and you get instant parallelism.
Based on eventlet, multiprocessing and redis.
It is ideal for executing multiple network bound tasks in parallel from a single node, without going through the pain of setting up a map reduce cluster. It uses both processes and green threads to extract the maximum out of a single node setup.
Installation
-----------
1. Install redis and start the server, **tasks** uses redis for queueing jobs. If you already have a redis server setup, call `tasks.set_redis` and pass a redis connection object with a different database/namespace from what you normally use in your application.
2. Install the redis-py and eventlet libraries.
`pip install redis eventlet`
3. Install tasks or copy this package to your source code.
`pip install tasks-py`
Usage
-----
Import `tasks` and call eventlet's monkey patch function in the first line of your module. Call `tasks.set_func` to register your function. This function will be receiving a string as an argument and its return value will be ignored. To indicate failure of the task, raise an error or exception within the function. Call `tasks.main()` to get the interactive command line options.
import eventlet
eventlet.monkey_patch()
import tasks
from urllib2 import urlopen
def fetch(url):
f = open('/tmp/download', 'w')
body = urlopen(url).read()
f.write(body)
f.close()
tasks.set_func(fetch)
tasks.main()
Now to add jobs, create a file with one argument per line and use this command.
`$ python yourfile.py add <list_of_jobs.txt>`
To start (or restart) the job processing (do this in a **screen** session or close the input stream):
`$ python yourfile.py run`
**tasks** has resume support, so it will start where you left off the last time.
To view the current status while it is running:
`$ python yourfile.py status`
Once you are done, you can clear the logs and the completed tasks by calling reset.
`$ python yourfile.py reset`
See the code or the test.py file for more information. Feel free to fork and modify this.
----
**Author** : Vivek Narayanan <<vivek@vivekn.com>>
**License** : BSD
(C) Vivek Narayanan (2014)
Metadata
Release files for tasks-py 1.0.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| tasks-py-1.0.3.tar.gz | 3.5 kB | Without provenance | Details for tasks-py-1.0.3.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| tasks-py-1.0.3.macosx-10.10-intel.tar.gz | Details for tasks-py-1.0.3.macosx-10.10-intel.tar.gz |
Total release size: 8.3 kB
Release files / tasks-py-1.0.3.tar.gz
| Download URL | tasks-py-1.0.3.tar.gz |
|---|---|
| Size | 3.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c2c162c7e20afcaddb0bb24012d4b58182f1ebfa36f3bc2e4d1f02484dc22784
|
|
BLAKE2b-256 checksum How to use checksums |
f31b43e081533932f54d09e2c1e1b7efb3b04d658bf28654d8132b276f4482d0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Release files / tasks-py-1.0.3.macosx-10.10-intel.tar.gz
| Download URL | tasks-py-1.0.3.macosx-10.10-intel.tar.gz |
|---|---|
| Size | 4.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c9c2d68c1be8933fa0e7fbe7c329c4c5dbd5b36038beca7f3f8180eb58eee6bf
|
|
BLAKE2b-256 checksum How to use checksums |
4abfc7b87d36fdaab6fd28111d80a626b4d2baf09e214cac6892634e451f2be5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Security for tasks-py 1.0.3
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.