Skip to main content

Worker decorator for background tasks re-using ThreadPoolExecutor.

Project description

# Soon
Worker decorator for background tasks re-using [ThreadPoolExecutor](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor).

Installation
------------

Soon is conveniently available via pip:
```
pip install soon
```

or installable via `git clone` and `setup.py`
```
git clone git@github.com:dotpot/Soon.git
sudo python setup.py install
```

To ensure Soon is properly installed, you can run the unittest suite from the project root:
```
pipenv run pytest -v
```

Usage
-----
The Soon library enables you to utilize the benefits of multi-threading with minimal concern about the implementation details.


Website fetcher example
-----------------
You've collected a list of urls and are looking to download the HTML of the lot. The following is a perfectly reasonable first stab at solving the task.

```python
urls = [
'https://cnn.com',
'https://news.ycombinator.com/',
'https://stackoverflow.com/',
]
```
---
```python
import time
import requests

def fetch(url):
return requests.get(url)

if __name__ == "__main__":

start = time.time()
responses = [fetch(url) for url in urls]
html = [response.text for response in responses]
end = time.time()
print("Time: %f seconds" % (end - start))
```
---

More efficient website fetcher example
--------------------------

Using Soon's decorator syntax, we can define a function that executes in multiple threads. Individual calls to `download` are non-blocking, but we can largely ignore this fact and write code identically to how we would in a synchronous paradigm.

```python
import time
import requests

from soon import workers

@workers(5)
def fetch(url):
return requests.get(url)

if __name__ == "__main__":
start = time.time()
responses = [fetch(url) for url in urls]
html = [response.text for response in responses]
end = time.time()
print("Time: %f seconds" % (end - start))

```
We can now download websites more efficiently.

---

You can also optionally pass in `timeout` argument, to prevent hanging on a task that is not guaranteed to return.

```python
import time

from soon import workers

@workers(1, timeout=0.1)
def timeout_error():
time.sleep(1)

if __name__ == "__main__":
timeout_error()
```


Project details


Release history Release notifications | RSS feed

This version

0.2

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for soon, version 0.2
Filename, size File type Python version Upload date Hashes
Filename, size soon-0.2-py3-none-any.whl (3.6 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size soon-0.2.tar.gz (2.6 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page