unctl 0.9.1

A health check and RCA tool for kubernetes

unctl

Table of Contents

1. About The Project
   • Checks
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
   • Config file
3. Usage
4. Roadmap
5. Contact

About The Project

unctl is a versatile command-line tool designed to perform a wide range of checks and inspections on various components of your infrastructure. It provides a unified interface to assess the health and performance of different services and platforms, and goes beyond mere diagnosis. With built-in AI capabilities, it guides you seamlessly from system diagnostic to remediation, offering intelligent solutions to address any issues it detects.

This addition emphasizes the tool's capacity to not only identify problems but also provide AI-driven recommendations and solutions for resolving those issues, making it even more valuable for infrastructure management and maintenance.

Checks

Provider	Checks
k8s	31
mysql	1

k8s checks

Check	ID	Service	Category	Severity	Description
Check if a k8s PVC is in Pending state	K8S401	pvc	Health	Critical	Alerts on pending PVCs, highlighting potential delays in provisioning persistent volume claims for all the namespaces
Check if the k8s node is in Ready state	K8S504	node	Health	Critical	Ensure node health by examining readiness conditions, signaling failures if any issues are detected in the node's status
Deployment has insufficient replicas	K8S801	deployment	Health	Critical	Validate Deployments for the correct number of available replicas, highlighting any discrepancies between desired and available counts
Pod has a high restart count	K8S104	pod	Health	Critical	Identify pods for all the namespaces where certain containers have restarted more than 10 times, indicating potential instability concerns
Pod is in CrashLoopBackOff state	K8S103	pod	Health	Critical	Identify pods with containers stuck in a CrashLoopBackOff state, highlighting potential issues impacting pod stability for all the namespaces
Service has endpoints that are NotReady	K8S304	service	Health	Severe	Highlights when services have NotReady endpoints, indicating potential disruptions to service reliability for all the namespaces
Service has no endpoints	K8S301	service	Health	Severe	Identify services with no associated endpoints, highlighting potential misconfigurations impacting service connectivity
Analyzing HPAs, checking if scale targets exist and have resources	K8S101	pod	HPA	High	Analyze optimal Horizontal Pod Autoscaler (HPA) configurations by ensuring associated resources (Deployments, ReplicationControllers, ReplicaSets, StatefulSets) have defined resource limits for effective auto-scaling
Check for the existence of Ingress class, service and secrets for all the namespaces	K8S201	ingress	Ingress	High	Ensure proper Ingress configurations by validating associated services, secrets, and ingress classes, flagging issues if there are missing elements or misconfigured settings for all the namespaces
Check the existence of secret in Daemonset	K8S603	daemonset	Daemonset, Secret	High	Ensure the presence of referenced Secrets in Daemonset volumes, reporting failures for any missing Secret within all the namespaces
Check the existence of secret in Deployment	K8S701	secret	Deployment	High	Ensure the presence of referenced Secrets in Deployment volumes, reporting failures for any missing Secret for all the namespaces
Excessive Pods on Node	K8S501	node	Resource Limits	High	Assesses nodes for excessive pod counts, flagging potential issues if pods near capacity thresholds based on CPU and memory resources
Find Deployments with missing configmap	K8S901	configmap	Deployment	High	Ensure the presence of referenced ConfigMaps in Deployment volumes, reporting failures for any missing ConfigMap for all the namespaces
Find Pending Pods	K8S106	pod	Health	High	Ensure that Pods are not in a Pending state due to scheduling issues or container creation failures, and report relevant details for diagnostics
Find Pods with missing configmap	K8S102	pod	Pod, ConfigMap	High	Ensure the presence of referenced ConfigMaps in Pod containers and volumes, reporting failures for any missing ConfigMap for all the namespaces
Find Pods with missing secrets	K8S105	pod	Pod, Secret	High	Ensure the presence of referenced Secrets in Pod containers, reporting failures for any missing Secret for all the namespaces
Insufficient PIDs on Node	K8S502	node	Performance	High	Check if the nodes have remaining PIDs less than a set threshold
Kubernetes Node Out-of-Memory Check	K8S503	node	Performance	High	Checks if any Kubernetes node is using more than 85% of its memory capacity.
Validate configmap existence in Statefulset	K8S1001	statefulset	StatefulSet	High	Ensure the existence of referenced ConfigMaps in StatefulSet volume claims and template volumes, reporting failures for any missing ConfigMap for all the namespaces
Validate cronjob starting deadline	K8S1101	cronjob	CronJob	High	Ensure CronJobs have a non-negative starting deadline, reporting failures for negative values for all the namespaces
Validate existence of configmaps in daemonsets	K8S601	daemonset	DaemonSet, ConfigMap	High	Ensure the presence of referenced ConfigMaps in Daemonset volumes, reporting failures for any missing ConfigMap for all the namespaces
Verify StatefulSet has valid service	K8S1002	statefulset	StatefulSet	High	Verify StatefulSet's service reference, ensuring it points to an existing service in all the namespaces, reporting failures for non-existent services
Verify StatefulSet has valid storageClass	K8S1003	statefulset	StatefulSet	High	Validate StatefulSet's storage class, ensuring it references existing storage classes in the namespace, reporting failures for non-existent ones
Zero Scale Deployment Check	K8S802	deployment	Availability	High	Verify that Deployments have a non-zero replica count, preventing unintentional scaling down to zero
Check if Kubernetes services have matching pod labels	K8S302	service	Configuration	Medium	This check validates if Kubernetes service selectors match pod labels. This ensures proper routing & discovery of pods.
Pod template validation in DaemonSet	K8S602	daemonset	Resource Management	Medium	Checks that the Pod template within a DaemonSet is configured correctly according to certain threshold values.
Services Target Port Match	K8S303	service	Diagnostic	Medium	This check identifies service ports that do not match their target ports
Validate that network policies are in place and configured correctly	K8S1201	networkpolicy	Network Security	Medium	Verify Network Policy configurations, highlighting issues if policies allow traffic to all pods or if not applied to any specific pods
Zero scale detected in statefulset	K8S1004	statefulset	Availability	Medium	Check to ensure that no StatefulSets are scaled to zero as it might hamper availability.
Find unused DaemonSet	K8S604	daemonset	DaemonSet, Cost, Resource Optimization	Low	Any DaemonSet that has been created but has no associated pods and remained unused for over 30 days.
Validate cronjobs schedule and state	K8S1102	cronjob	CronJob	Low	Ensure CronJobs have valid schedules and are not suspended, reporting failures for any invalid schedules or suspended jobs for all the namespaces

mysql checks

Check	ID	Service	Category	Severity	Description
Checks max used connections	MYSQL101	global	Connection, Thread	High	Checks max used connections reaching max count

Built With

Getting Started

Prerequisites

• Python >= 3.10
• OpenAI API Key - to have AI based functionality enabled

Installation

1. Get distibution on your machine:
   • Run pip command to install unctl from PyPI

     pip install unctl
     

2. (optional) Set OpenAI API key to be able to use --explain (-e) option

   export OPENAI_API_KEY=<your api key>
   

Kubernetes

1. (optional) Set KUBECONFIG variable to specific location other than default:

   export KUBECONFIG=<path to kube config file>
   

2. Run unctl command to see list of options:

   unctl k8s -h
   

MySQL

1. unctl is using ~/.my.cnf as config path.
2. Run unctl command to see list of options:

   unctl mysql -h
   

Config file

By default unctl is looking at ~/.config/unctl/config.yaml. Otherwise it would use default values.

To specify path to the config file use --config option:

unctl --config <path to file> {provider} ...

Note: CLI options will overwrite values in config file

Anonymisation

This section allows to handle data manipulation sending to 3rd party services in order to hide PII or any other sensitive data:

• masks - list of rules to mask data. Can be extended with own regex patterns. By default there are 2 rules present in the config: email and ip_address.

Example:

anonymisation:
  masks:
    - name: email
      pattern: \b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b
    - name: ip_address
      pattern: \b(?:\d{1,3}\.){3}\d{1,3}\b

Filter

This section allows to set filters for your scan options:

• failed_only - shows only failed checks in the report when true. Default is false
• checks - list of checks IDs to run scan for specific checks only. If value is [] or missed than filter considered as disabled. Default is []
• categories - list of checks categories to run scan. If value is [] or missed than filter considered as disabled. Default is []
• services - list of checks services to run scan. If value is [] or missed than filter considered as disabled. Default is []

Example:

filter:
  failed_only: false
  checks:
    - K8S101
    - K8S203
  categories:
    - Health
  services:
    - pod
    - deployment

Interactive mode

Section is reponsible for interactive mode configuration:

• prompt - defines whether ask user to enter in the interactive mode after report is done. Default is true

Example:

interactive:
  prompt: true

Muting

Section is taking care of muting particular checks and objects:

• checks - list of check IDs which will be ignored by the scan. If value is [] or missed than filter considered as disabled. Default is []
• objects - set of objects with list of specific check IDs which should be ignored. If value is {} or missed than filter considered as disabled. Default is {}

Example:

ignore:
  checks: 
    - K8S101
  objects:
    "some-object-name-1": [] # ignore object for all checks
    "some-object-name-2":    # ignore object for the specific checks
      - K8S203

Usage

unctl

% unctl -h
usage: unctl [-h] [-v] [--config CONFIG [CONFIG ...]] {k8s,mysql} ...

          Welcome to unSkript CLI Interface 

options:
  -h, --help            show this help message and exit
  -v, --version         show program`s version number and exit
  --config CONFIG [CONFIG ...]
                        Specify path to the unctl config file

unctl available providers:
  {k8s,mysql}

To see the different available options on a specific provider, run:
    unctl {provider} -h|--help

Provider

% unctl {provider} -h
usage: unctl k8s [-h] [-s] [-e] [-f] [-c CHECKS [CHECKS ...]] [--sort-by {object,check}] [--categories CATEGORIES [CATEGORIES ...]] [--services SERVICES [SERVICES ...]] [-l]
                 [--no-interactive] [--list-categories] [--list-services] [-r] [--ignore IGNORE [IGNORE ...]] [--ignore-objects IGNORE_OBJECTS [IGNORE_OBJECTS ...]]

options:
  -h, --help            show this help message and exit
  -s, --scan            Run a provider scan
  -e, --explain         Explain failures using AI
  -f, --failing-only    Show only failing checks
  -c CHECKS [CHECKS ...], --checks CHECKS [CHECKS ...]
                        Filter checks by IDs
  --sort-by {object,check}
                        Sort results by 'object' (default) or 'check'
  --categories CATEGORIES [CATEGORIES ...]
                        Filter checks by category
  --services SERVICES [SERVICES ...]
                        Filter checks by services
  -l, --list-checks     List available checks
  --no-interactive      Interactive mode is not allowed. Prompts will be skipped
  --list-categories     List available categories
  --list-services       List available services
  -r, --remediate       Create remediation plan
  --ignore IGNORE [IGNORE ...]
                        Ignoring one or more checks by ID
  --ignore-objects IGNORE_OBJECTS [IGNORE_OBJECTS ...]
                        Ignoring one or more objects by name

Roadmap

• K8s checks - in progress
• MySQL checks - in progress
• Elastic Search checks
• AWS checks
• GCP checks

Contact

Abhishek Saxena: [e-mail]

Official website: https://unskript.com/