Delay Generators

Different on_delay generators can be used for increasing the delays between successive retries. Note that the values for the delays are given in milliseconds.

@rerun(on_delay=[1000, 2000], on_error=KeyError)
def func():
    ...

Generators and iterable items can be used to generate delays too.

def fancy_generator():
    # yield delays
    ...

@rerun(on_delay=fancy_generator)
def func():
    ...

If a single delay is desired, an integer or float value can be given, like so.

@rerun(on_delay=1000, on_error=KeyError)
def func():
    ...

A couple of generator functions are provided in the library. These are the typical algorithms used in most systems, and can serve as a baseline example for more complex delay systems.

• constant(delay, limit): yields a constant delay at each iteration

• linear(start, increment, limit): yields a linearly increasing delay at each iteration

• exponential(base, multiplier, limit): yields an exponentially increasing delay at each iteration

• fibonacci(multiplier, limit): yields a delay following the fibonacci pattern at each iteration

If the function fails to yield a response that isn’t handled before running out of generated items by the on_delay generator, a MaxRetryException is thrown.

@rerun(on_delay=None, on_error=KeyError)  # No retries
def func():
    raise KeyError

# MaxRetryException is raised

Error Handling

The on_error can be used to determine if a raised exception should be handled and the function retried. A single exception can be specified to be handled. If an exception is raised that isn’t handled, it will bubble up to the outer scope without retrying the function.

@rerun(on_delay=[1000], on_error=TypeError)
def func():
    raise KeyError

# KeyError isn't handled, and is thus raised

Multiple errors can be given as a sequence to handle more than one.

@rerun(on_delay=[1000], on_error=[ValueError, TimeoutError])
def func():
    ...

A callable object (such as a function), can be used for more complex handling of errors. These should accept a single value, the error raised, and return a boolean indicating True to handle, or False to not.

@rerun(on_delay=[1000], on_error=lambda x: not isinstance(ValueError, TimeoutError))
def func():
    ...

Return Value Handling

Like raised exception, return values can also be handled in a similar manner. Return values that are handled cause the function to be retried, and those that aren’t are simply return. A common use case for this is when interacting with functions that yield a return value that indicates a failed state (like -1 or None), while other values indicate a successful state (like 0 or an object).

@rerun(on_delay=[1000], on_return=-1)
def func()
    return -1

# Function is retried because -1 is handled

One note to make is that if a sequence is given, any value that is matched in the sequence is handled. If, however, the return value is a sequence, either a function should be used to check for equality or on_return should be a sequence of sequences, like so.

# WRONG: checks if [-1, -1] is in the sequence [-1, -1]
@rerun(on_delay=[1000], on_return=[-1, -1])
def func():
    return [-1, -1]  # Not handled

# CORRECT: checks if [-1, -1] is the return value
@rerun(on_delay=[1000], on_return=lambda x: x == [-1, -1])
def func():
    return [-1, -1] # Is handled

# CORRECT: checks if [-1, -1] is in the sequence [[-1, -1]]
@rerun(on_delay=[1000], on_return=[[-1, -1]])
def func():
    return [-1, -1] # Is handled

Each time a retry takes place the on_retry callback is called, if given, passing in the current delay and the number of retries thus far. Logging is a common use-case for this, as shown below.

def log(delay, retry):
    logging.info('Retrying function again (#%s) in %s seconds' % (delay, retry))

@rerun(on_delay=[1000, 2000, 3000], on_return=-1, on_retry=log)
def func():
    ...

The on_retry callback is called prior to waiting for the delay in-between successive retries. If calling the it after the delay, the retry_after_delay parameter can be specified.

@rerun(on_delay=[1000],
         on_return=-1,
         on_retry=lambda d, r: print('Waited %s seconds for retry #%s' % (d, r)))
def func():
    ...