Skip to main content

Worker decorator for background tasks re-using ThreadPoolExecutor.

Project description

# Soon
Worker decorator for background tasks re-using [ThreadPoolExecutor](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor).

Installation
------------

Soon is conveniently available via pip:
```
pip install soon
```

or installable via `git clone` and `setup.py`
```
git clone git@github.com:dotpot/Soon.git
sudo python setup.py install
```

To ensure Soon is properly installed, you can run the unittest suite from the project root:
```
pipenv run pytest -v
```

Usage
-----
The Soon library enables you to utilize the benefits of multi-threading with minimal concern about the implementation details.


Website fetcher example
-----------------
You've collected a list of urls and are looking to download the HTML of the lot. The following is a perfectly reasonable first stab at solving the task.

```python
urls = [
'https://cnn.com',
'https://news.ycombinator.com/',
'https://stackoverflow.com/',
]
```
---
```python
import time
import requests

def fetch(url):
return requests.get(url)

if __name__ == "__main__":

start = time.time()
responses = [fetch(url) for url in urls]
html = [response.text for response in responses]
end = time.time()
print("Time: %f seconds" % (end - start))
```
---

More efficient website fetcher example
--------------------------

Using Soon's decorator syntax, we can define a function that executes in multiple threads. Individual calls to `download` are non-blocking, but we can largely ignore this fact and write code identically to how we would in a synchronous paradigm.

```python
import time
import requests

from soon import workers

@workers(5)
def fetch(url):
return requests.get(url)

if __name__ == "__main__":
start = time.time()
responses = [fetch(url) for url in urls]
html = [response.text for response in responses]
end = time.time()
print("Time: %f seconds" % (end - start))

```
We can now download websites more efficiently.

---

You can also optionally pass in `timeout` argument, to prevent hanging on a task that is not guaranteed to return.

```python
import time

from soon import workers

@workers(1, timeout=0.1)
def timeout_error():
time.sleep(1)

if __name__ == "__main__":
timeout_error()
```


Project details


Release history Release notifications | RSS feed

This version

0.2

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

soon-0.2.tar.gz (2.6 kB view hashes)

Uploaded Source

Built Distribution

soon-0.2-py3-none-any.whl (3.6 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page