Retries for humans
Opnieuw is a general-purpose retrying library, written in Python, in order to simplify the task of adding retry behavior to both synchronous as well as asynchronous tasks. Opnieuw is easy and straightforward to use.
Opnieuw makes it easy to add robust retries:
- There is a single retry strategy, exponential backoff with jitter, which makes it impossible to forget to add jitter.
- It has just two parameters, to eliminate the possibility of invalid and nonsensical combinations.
- Parameters are named clearly, to avoid confusing e.g. number of calls (including the initial one) with number of retries (excluding the initial call).
- The parameters are intuitive: instead of configuring the base delay for exponential backoff, Opnieuw accepts a maximum number of calls to make, and maximum time after the first call to still retry.
- Time units are clear from parameter names to make the decorated code obvious, and readable without needing to refer to documentation.
See our announcement post for a bit more background on why we wrote Opnieuw.
Suppose we want to parse
https://tech.channable.com/atom.xml and we want to
add a retry to handle a specific network Exception. We can add Opnieuw to our
network handler as follows:
import requests from requests.exceptions import ConnectionError, HTTPError from opnieuw import retry @retry( retry_on_exceptions=(ConnectionError, HTTPError), max_calls_total=4, retry_window_after_first_call_in_seconds=60, ) def get_page() -> str: response = requests.get('https://tech.channable.com/atom.xml') return response.text
retry_on_exceptionsspecifies which exceptions we want to retry on.
max_calls_totalis the maximum number of times that the decorated function gets called, including the initial call.
retry_window_after_first_call_in_secondsis the maximum number of seconds after the first call was initiated, where we would still do a new attempt.
- Generic decorator API
- Specify retry exception (i.e. type of exception that we want retry)
- Exponential backoff with jitter
- Pre-shipped list of popular exceptions, which can easily be expanded
To install Opnieuw, simply:
$ pip install opnieuw
The short example above provides a concise demonstration of how Opnieuw could
be used. Let's dig deeper into Opnieuw and add another exception to
retry_on_exceptions to do a retry on:
from urllib.error import URLError import requests from opnieuw import RetryException, retry @retry( retry_on_exceptions=(ConnectionError, RetryException, URLError), max_calls_total=4, retry_window_after_first_call_in_seconds=60, ) def get_page() -> str: response = requests.get('https://tech.channable.com/atom.xml') if response.status_code != 200: raise RetryException return response.text
We can pass the name of exception we want do retry for or a tuple of exceptions
retry_on_exceptions. As mentioned earlier Opnieuw was developed to
make it more convenient to add retry behavior to any task.
Let's make it a little bit more generic and define a list of retry exceptions that should trigger a retry of the function:
STANDARD_HTTP_EXCEPTIONS = ( ConnectionError, ProtocolError, RetryException, ... ) @retry( retry_on_exceptions=STANDARD_HTTP_EXCEPTIONS, max_calls_total=4, retry_window_after_first_call_in_seconds=60, ) def get_page() -> str: response = requests.post('https://tech.channable.com/atom.xml') return response.text
Now our retry is more generic, as exceptions raised which are in
STANDARD_HTTP_EXCEPTIONS will be retried.
If you want retry behavior for async tasks, then there is also an async retry which basically work the same way, but for async tasks.
Here is the example above but in async mode:
from opnieuw import retry_async STANDARD_HTTP_EXCEPTIONS = ( ConnectionError, EOFError, RetryException, ... ) @retry_async( retry_on_exceptions=STANDARD_HTTP_EXCEPTIONS, max_calls_total=4, retry_window_after_first_call_in_seconds=60, ) async def get_page() -> str: response = requests.get('https://tech.channable.com/atom.xml') return response.text
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size opnieuw-1.1.0-py3-none-any.whl (8.9 kB)||File type Wheel||Python version py3||Upload date||Hashes View|
|Filename, size opnieuw-1.1.0.tar.gz (6.6 kB)||File type Source||Python version None||Upload date||Hashes View|