pycircuitbreaker 0.5.0

Python implementation of the Circuit Breaker Pattern

PyCircuitBreaker

Python Implementation of the Circuit Breaker Pattern. Inspired by circuitbreaker by Fabian Fuelling.

Installation

pip install pycircuitbreaker

Usage

The simplest usage of pycircuitbreaker is to decorate a function that can fail using circuit.

from pycircuitbreaker import circuit

@circuit
def function_that_can_fail():
    ...

For readiness probes (PaaS, k8s, ...) it is common to expose the different circuit breakers state.

from pycircuitbreaker import circuit, CircuitBreakerRegistry


registry = CircuitBreakerRegistry()


@app.route("/ready")
def ready():
    content = {
        "circuits": {
            cb.id: cb.state.name for cb in registry.get_circuits()
        }
    }
    status = (
        200 if len(list(registry.get_open_circuits())) == 0 else 500
    )
    return content, status, {"Cache-Control": "no-cache"}

Note that the registry is not automatically managed by the library, it is the application responsibility to register created circuit breakers.

It is also possible to reuse the same circuit breaker for different functions that rely on the same external dependency.

def db_breaker(func: Callable) -> Callable:
    breaker = CircuitBreaker(
        breaker_id="db",
        error_threshold=5,
        recovery_timeout=30,
        recovery_threshold=1,
        exception_denylist=[DisconnectionError, TimeoutError],
    )

    @wraps(func)
    def wrapper(*args: Any, **kwargs: Any) -> Any:
        return breaker.call(func, *args, **kwargs)

    return circuit_wrapper

@db_breaker
def call_to_db():
    ...

@db_breaker
def another_call_to_db():
    ...

Complete usage example

from pycircuitbreaker import CircuitBreakerRegistry, CircuitBreaker


registry = CircuitBreakerRegistry()


@app.route("/ready")
def ready():
    content = {
        "circuits": {
            cb.id: cb.state.name for cb in registry.get_circuits()
        }
    }
    status = (
        200 if len(list(registry.get_open_circuits())) == 0 else 500
    )
    return content, status, {"Cache-Control": "no-cache"}


def db_breaker(func: Callable) -> Callable:
    breaker = CircuitBreaker(
        breaker_id="db",
        error_threshold=5,
        recovery_timeout=30,
        recovery_threshold=1,
        exception_denylist=[DisconnectionError, TimeoutError],
    )
    registry.register(breaker)

    @wraps(func)
    def wrapper(*args: Any, **kwargs: Any) -> Any:
        return breaker.call(func, *args, **kwargs)

    return circuit_wrapper


@db_breaker
def call_to_db():
    ...


@db_breaker
def another_call_to_db():
    ...

Reset Strategies

By default, pycircuitbreaker operates such that a single success resets the error state of a closed breaker. This makes sense for a service that rarely fails, but in certains cases this can pose a problem. If the error_threshold is set to 5, but only 4/5 external requests fail, the breaker will never open. To get around this, the strategy setting may be used. By setting this to pycircuitbreaker.CircuitBreakerStrategy.NET_ERROR, the net error count (errors - successes) will be used to trigger the breaker.

Configuration

A number of configuration options can be provided to the CircuitBreaker class or the circuit decorator to control the behaviour of the breaker. When using the decorator, options should be passed as keyword arguments.

breaker_id

The ID of the breaker used in exception reporting or for logging purposes. If not specified, a uuid4() is created.

detect_error

Type: Optional[Callable[Any, bool]]

This option can be used to detect errors that do not raise exceptions. For example, if you have a function that returns a response object with a status code, we can detect errors that have a status code of 500.

from pycircuitbreaker import circuit

def detect_500(response) -> bool:
    return response.status_code == 500

@circuit(detect_error=detect_500)
def request():
    response = external_call()
    return response

error_threshold

Type: Optional[int] Default: 5

The number of sequential errors that must occur before the breaker opens. If 4 errors occur a single success will reset the error count to 0.

exception_denylist

Type: Optional[Iterable[Exception]]

There are cases where only certain errors should count as errors that can open the breaker. In the example below, we are using requests to call to an external service and then raise an exception on an error case. We only want the circuit breaker to open on timeouts to the external service.

Note that if this option is used, errors derived from those specified will also be included in the denylist.

import requests
from pycircuitbreaker import circuit

@circuit(exception_denylist=[requests.exceptions.Timeout])
def external_call():
    response = requests.get("EXTERNAL_SERVICE")
    response.raise_for_status()

exception_allowlist

Type: Optional[Iterable[Exception]]

This setting allows certain exceptions to not be counted as errors. Taking the same example as the exception_denylist setting, we can ignore request.exceptions.HTTPError only using the allowlist.

Note that if this option is used, errors derived from those specified will also be included in the allowlist.

import requests
from pycircuitbreaker import circuit

@circuit(exception_allowlist=[requests.exceptions.HTTPError])
def external_call():
    response = requests.get("EXTERNAL_SERVICE")
    response.raise_for_status()

on_close

Type: Optional[Callable[[CircuitBreaker], None]]

If specified, this function is called when the breaker fully closes. This can be useful for logging messages.

on_open

Type: Optional[Callable[[CircuitBreaker, Union[Exception, Any]], None]]

If specified, this function is called when the breaker opens. The 2nd parameter to the function will be the exception that triggered the opening if exception detection was used. If the detect_error method was used, the wrapped function return value is passed as the 2nd parameter.

recovery_threshold

Type: Optional[int] Default: 1

This is the number of successful calls that must occur before the circuit breaker transitions from CircuitBreakerState.HALF_OPEN to CircuitBreakerState.CLOSED.

recovery_timeout

Type: Optional[int] Default: 30

The number of seconds the breaker stays fully open for before test requests are allowed through.

strategy

Type: Optional[pycircuitbreaker.CircuitBreakerStrategy] Default: pycircuitbreaker.CircuitBreakerStrategy.SINGLE_RESET

Controls how successes change the error count when the breaker is closed. By default, a single success resets the number of errors in the breaker.

Possible options:

• CircuitBreakerStrategy.SINGLE_RESET
• CircuitBreakerStrategy.NET_ERROR

CircuitBreaker API

The public API of the CircuitBreaker class is described below.

error_count

Type: int

The number of errors stored in the breaker.

id

The ID of the breaker. If not supplied via the configuration breaker_id setting, this is a uuid4().

open_time

Type: datetime

The UTC time the breaker last opened.

recovery_start_time

Type: datetime

The UTC time that the breaker is open until (when recovery begins).

state

Type: CircuitBreakerState

The state of the breaker.

success_count

Type: int

The number of successes stored in the breaker during the recovery period.

Roadmap

1. Back circuit breaker state with Redis to share state among processes (e.g. for gunicorn)