Skip to main content

Worker decorator for background tasks re-using ThreadPoolExecutor.

Project description

# Soon
Worker decorator for background tasks re-using [ThreadPoolExecutor](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor).

Installation
------------

Soon is conveniently available via pip:
```
pip install soon
```

or installable via `git clone` and `setup.py`
```
git clone git@github.com:dotpot/Soon.git
sudo python setup.py install
```

To ensure Soon is properly installed, you can run the unittest suite from the project root:
```
pipenv run pytest -v
```

Usage
-----
The Soon library enables you to utilize the benefits of multi-threading with minimal concern about the implementation details.


Website fetcher example
-----------------
You've collected a list of urls and are looking to download the HTML of the lot. The following is a perfectly reasonable first stab at solving the task.

```python
urls = [
'https://cnn.com',
'https://news.ycombinator.com/',
'https://stackoverflow.com/',
]
```
---
```python
import time
import requests

def fetch(url):
return requests.get(url)

if __name__ == "__main__":

start = time.time()
responses = [fetch(url) for url in urls]
html = [response.text for response in responses]
end = time.time()
print("Time: %f seconds" % (end - start))
```
---

More efficient website fetcher example
--------------------------

Using Soon's decorator syntax, we can define a function that executes in multiple threads. Individual calls to `download` are non-blocking, but we can largely ignore this fact and write code identically to how we would in a synchronous paradigm.

```python
import time
import requests

from soon import workers

@workers(5)
def fetch(url):
return requests.get(url)

if __name__ == "__main__":
start = time.time()
responses = [fetch(url) for url in urls]
html = [response.text for response in responses]
end = time.time()
print("Time: %f seconds" % (end - start))

```
We can now download websites more efficiently.

---

You can also optionally pass in `timeout` argument, to prevent hanging on a task that is not guaranteed to return.

```python
import time

from soon import workers

@workers(1, timeout=0.1)
def timeout_error():
time.sleep(1)

if __name__ == "__main__":
timeout_error()
```


Project details


Release history Release notifications

This version

0.2

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for soon, version 0.2
Filename, size File type Python version Upload date Hashes
Filename, size soon-0.2-py3-none-any.whl (3.6 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size soon-0.2.tar.gz (2.6 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page