Pod-autoscaling plugin for Tutor

This plugin enables Pod-Autoscaling strategies for instances deployed in Kubernetes with Tutor. Inspired by the implementation of HPA from https://gitlab.com/opencraft/dev/tutor-contrib-grove (thanks @gabor-boros) The strategies offered by the plugin are:

1. HPA (Horizontal Pod Autoscaler): this mechanism adds or removes pods based on a defined metric threshold (For instance CPU or memory consumption).
2. VPA (Vertical Pod Autoscaler): this strategy aims to stabilize the consumption and resources of every pod, so they're kept between limits and requests that were specified in the initial pod configuration.

Requirements

1. To use HPA, the installation of metrics-server is required.
2. To use VPA, the installation of Vertical Pod Autoscaler is required.

Installation

pip install tutor-contrib-pod-autoscaling

Configuration

This plugin implements a filter called AUTOSCALING_CONFIG (tutorpod_autoscaling.hooks.AUTOSCALING_CONFIG) which allow to add/modify pod autoscaling configuration for different OpenedX services. The plugin by itself uses the AUTOSCALING_CONFIG filter to add default autoscaling configuration (HPA and VPA) for the LMS, CMS, LMS_WORKER and CMS_WORKER deployments based on CPU and MEMORY metrics (check the CORE_AUTOSCALING_CONFIG variable in the plugin.py file).

Adding/changing HPA/VPA configuration for OpenedX services

Operators can take advantage of this plugin to configure their HPA/VPA settings for different services. There are 2 mechanisms to do so:

1. Create a Tutor plugin and add your HPA/VPA configuration to the tutorpod_autoscaling.hooks.AUTOSCALING_CONFIG filter. For instance, to add HPA support to the forum deployment:

from tutorpod_autoscaling.hooks import AUTOSCALING_CONFIG


@AUTOSCALING_CONFIG.add()
def _add_my_autoscaling(autoscaling_config):
    autoscaling_config["forum"] = {
        "enable_hpa": True,
        "memory_request": "300Mi",
        "cpu_request": 0.25,
        "memory_limit": "1200Mi",
        "cpu_limit": 1,
        "min_replicas": 1,
        "max_replicas": 10,
        "avg_cpu": 300,
        "avg_memory": "",
        "enable_vpa": False,
        "behavior": {},
    }
    return autoscaling_config

You can also override the HPA/VPA configuration for any of the services supported by default, for instance, LMS:

from tutorpod_autoscaling.hooks import AUTOSCALING_CONFIG


@AUTOSCALING_CONFIG.add()
def _add_my_autoscaling(autoscaling_config):
    autoscaling_config["lms"] = {
        "enable_hpa": True,
        "memory_request": "1Gi",
        "cpu_request": 0.4,
        "memory_limit": "2Gi",
        "cpu_limit": 1,
        "min_replicas": 5,
        "max_replicas": 20,
        "avg_cpu": 70,
        "avg_memory": "",
        "enable_vpa": False,
        "behavior": {},
    }
    return autoscaling_config

2. Set the POD_AUTOSCALING_EXTRA_SERVICES variable to extend HPA/VPA support to different services of modify default ones:

POD_AUTOSCALING_EXTRA_SERVICES:
    forum:
        enable_hpa: true
        memory_request: 300Mi
        cpu_request: 0.25
        memory_limit: 1200Mi
        cpu_limit: 1
        min_replicas: 1
        max_replicas: 10
        avg_cpu: 300
        avg_memory: ''
        enable_vpa: true
        behavior: {}
    lms:
        enable_hpa: true
        memory_request: 1Gi
        cpu_request: 0.4
        memory_limit: 2Gi
        cpu_limit: 1
        min_replicas: 5
        max_replicas: 20
        avg_cpu: 70
        avg_memory: ''
        enable_vpa: true
        behavior: {}

Configuring HPA scaling behaviour

Set behavior to control HPA scale-up and scale-down dynamics. An empty dict ({}) omits the block, preserving Kubernetes defaults. See the Kubernetes HPA behaviour docs for all supported fields.

behavior:
  scaleDown:
    stabilizationWindowSeconds: 900
    policies:
    - type: Percent
      value: 10
      periodSeconds: 120

Migrating to Redwood version (18.x.x)

In versions prior to Redwood, the plugin used multiple configurations and a couple of patches to provide HPA/VPA support. Let's suppose you want to migrate to version 18.x.x and you have the following configuration in your config.yml for the LMS HPA/VPA support:

POD_AUTOSCALING_LMS_HPA: true
POD_AUTOSCALING_LMS_MEMORY_REQUEST: "350Mi"
POD_AUTOSCALING_LMS_CPU_REQUEST: 0.25
POD_AUTOSCALING_LMS_MEMORY_LIMIT: "1400Mi"
POD_AUTOSCALING_LMS_CPU_LIMIT: 1
POD_AUTOSCALING_LMS_MIN_REPLICAS: 1
POD_AUTOSCALING_LMS_MAX_REPLICAS: 4
POD_AUTOSCALING_LMS_AVG_CPU: 300
POD_AUTOSCALING_LMS_AVG_MEMORY: ""
POD_AUTOSCALING_LMS_VPA: false

The equivalent configuration for the 18.x.x version using the AUTOSCALING_CONFIG filter would be like this:

from tutorpod_autoscaling.hooks import AUTOSCALING_CONFIG


@AUTOSCALING_CONFIG.add()
def _add_my_autoscaling(autoscaling_config):
    autoscaling_config["lms"] = {
        "enable_hpa": True,
        "memory_request": "350Mi",
        "cpu_request": 0.25,
        "memory_limit": "1400Mi",
        "cpu_limit": 1,
        "min_replicas": 1,
        "max_replicas": 4,
        "avg_cpu": 300,
        "avg_memory": "",
        "enable_vpa": False,
        "behavior": {},
    }
    return autoscaling_config

The migration of other services follows the same logic.

It is important to mention that pod-autoscaling-hpa and pod-autoscaling-vpa patches were removed in the Redwood release since they are longer required in the HPA/VPA configuration model.

Notes to take in mind when using this plugin:

• The default values for HPA in this plugin can work OK for small installations. However, according to your use case, you'll need to tune the values in order to get the best performance.
• The VPA entities are configured to just display suggestions on the right amount of resources to allocate for every workload, and not to go directly and modify the resources allocated for a workload. This is because using HPA and VPA in automatic UpdateMode is not recommended. The best practice is to get the suggestions from the VPA and based on those suggestions, adjust the HPA values for the workloads in order to get the most value out of these autoscaling tools.

Usage

    tutor plugins enable pod-autoscaling

License

This software is licensed under the terms of the AGPLv3.