rerun.me 1.0.0

A library for rerunning erred functions with delays.

A library for rerunning functions in the case of raised exceptions and specific return values with configurable delays.

@rerun(on_delay=fibonacci(1000, 3),
         on_error=[ConnectionTimeoutError, DeadlockVictimError],
         on_return=[None]
         on_retry=lambda d, r: log.info('Retrying connection again (#%s) in %s seconds' % (r, d)))
def connection(conn_str, params):
    conn = db(conn_str, params)
    return db.open()

Installation

The latest version of rerun.me is available via pip:

pip install rerun.me

Alternatively, you can download and install from source:

python setup.py install

Getting Started

The rerun function contains the following signature:

def rerun(on_delay=None, on_error=None, on_return=None, on_retry=None, retry_after_delay=False):
    ...

It serves as both a function decorator, and a runnable wrapper and is configurable through it’s dynamic parameters. Most of which are function callbacks which allow the user to highly configure the retrying behavior.

This configurable nature is what sets this library apart from others with similar functionality. Many of which allow basic configuration using defined retry limits and constant delays between requests, which may be OK for the most simplistic of use cases. But most applications need more complex functionality which can delay with various common algorithms such as exponential or fibonacci delays. This library provides a subset of the most common delay generators, but is easily expandable to fit the application-specific needs.

Delay Generators

Different on_delay generators can be used for increasing the delays between successive retries. Note that the values for the delays are given in milliseconds.

@rerun(on_delay=[1000, 2000], on_error=KeyError)
def func():
    ...

Generators and iterable items can be used to generate delays too.

def fancy_generator():
    # yield delays
    ...

@rerun(on_delay=fancy_generator)
def func():
    ...

If a single delay is desired, an integer or float value can be given, like so.

@rerun(on_delay=1000, on_error=KeyError)
def func():
    ...

A couple of generator functions are provided in the library. These are the typical algorithms used in most systems, and can serve as a baseline example for more complex delay systems.

• constant(delay, limit): yields a constant delay at each iteration

• linear(start, increment, limit): yields a linearly increasing delay at each iteration

• exponential(base, multiplier, limit): yields an exponentially increasing delay at each iteration

• fibonacci(multiplier, limit): yields a delay following the fibonacci pattern at each iteration

If the function fails to yield a response that isn’t handled before running out of generated items by the on_delay generator, a MaxRetryException is thrown.

@rerun(on_delay=None, on_error=KeyError)  # No retries
def func():
    raise KeyError

# MaxRetryException is raised

Error Handling

The on_error can be used to determine if a raised exception should be handled and the function retried. A single exception can be specified to be handled. If an exception is raised that isn’t handled, it will bubble up to the outer scope without retrying the function.

@rerun(on_delay=[1000], on_error=TypeError)
def func():
    raise KeyError

# KeyError isn't handled, and is thus raised

Multiple errors can be given as a sequence to handle more than one.

@rerun(on_delay=[1000], on_error=[ValueError, TimeoutError])
def func():
    ...

A callable object (such as a function), can be used for more complex handling of errors. These should accept a single value, the error raised, and return a boolean indicating True to handle, or False to not.

@rerun(on_delay=[1000], on_error=lambda x: not isinstance(ValueError, TimeoutError))
def func():
    ...

Return Value Handling

Like raised exception, return values can also be handled in a similar manner. Return values that are handled cause the function to be retried, and those that aren’t are simply return. A common use case for this is when interacting with functions that yield a return value that indicates a failed state (like -1 or None), while other values indicate a successful state (like 0 or an object).

@rerun(on_delay=[1000], on_return=-1)
def func()
    return -1

# Function is retried because -1 is handled

One note to make is that if a sequence is given, any value that is matched in the sequence is handled. If, however, the return value is a sequence, either a function should be used to check for equality or on_return should be a sequence of sequences, like so.

# WRONG: checks if [-1, -1] is in the sequence [-1, -1]
@rerun(on_delay=[1000], on_return=[-1, -1])
def func():
    return [-1, -1]  # Not handled

# CORRECT: checks if [-1, -1] is the return value
@rerun(on_delay=[1000], on_return=lambda x: x == [-1, -1])
def func():
    return [-1, -1] # Is handled

# CORRECT: checks if [-1, -1] is in the sequence [[-1, -1]]
@rerun(on_delay=[1000], on_return=[[-1, -1]])
def func():
    return [-1, -1] # Is handled

Each time a retry takes place the on_retry callback is called, if given, passing in the current delay and the number of retries thus far. Logging is a common use-case for this, as shown below.

def log(delay, retry):
    logging.info('Retrying function again (#%s) in %s seconds' % (delay, retry))

@rerun(on_delay=[1000, 2000, 3000], on_return=-1, on_retry=log)
def func():
    ...

The on_retry callback is called prior to waiting for the delay in-between successive retries. If calling the it after the delay, the retry_after_delay parameter can be specified.

@rerun(on_delay=[1000],
         on_return=-1,
         on_retry=lambda d, r: print('Waited %s seconds for retry #%s' % (d, r)))
def func():
    ...

Advanced Usage

Instead of using as a decorator, rerun can be used as an instead for wrapping an arbitrary number of function calls. This can be achieved via the run method.

def func_a():
    ...

def func_b():
    ...

rerunner = rerun(on_delay=..., on_error=..., on_return=..., on_retry=...)

# Using same configured rerun instance
rerun.run(func_a, args, kwargs)
rerun.run(func_b, args, kwargs)

Besides using the provided run method, like any decorator functions can be locally wrapped, passed around, and executed.

def func():
    ...

rerunner = rerun(on_delay=..., on_error=..., on_return=..., on_retry=...)
rerun_func = rerunner(func)
rerun_func(args, kwargs)

# Or as a one-off like so
rerun(...)(func)(args, kwargs)

Each of the function parameters that can be passed into rerun, can actually be configured to accepts different number of parameters depending on the function. They can each either accept 0 parameters, the parameters that would be typically passed in, or the wrapped function’s args and kwargs in addition to the parameters typically given.

Optionally passing in the args and kwargs allows for building more complex callback functions. Each of the possible function variations are shown below.

def on_delay(): ...
def on_delay(*args, **kwargs): ...

def on_error(): ...
def on_error(error): ...
def on_error(error, *args, **kwargs): ...

def on_return(): ...
def on_return(value): ...
def on_return(value, *args, **kwargs): ...

def on_retry(): ...
def on_retry(delay, retries): ...
def on_retry(delay, retries, *args, **kwargs): ...

Contribution

Contributions or suggestions are welcome! Feel free to open an issue if a bug is found or an enhancement is desired, or even a pull request.

Changelog

All changes and versioning information can be found in the CHANGELOG.

License

Copyright (c) 2018 Jared Gillespie. See LICENSE for details.