unctl 0.8.0

A health check and RCA tool for kubernetes

unctl

Table of Contents

1. About The Project
   • Built With
   • Checks
2. Getting Started
   • Prerequisites
   • Installation
   • Config file
3. Usage
4. Roadmap
5. Contact

About The Project

unctl is a versatile command-line tool designed to perform a wide range of checks and inspections on various components of your infrastructure. It provides a unified interface to assess the health and performance of different services and platforms, and goes beyond mere diagnosis. With built-in AI capabilities, it guides you seamlessly from system diagnostic to remediation, offering intelligent solutions to address any issues it detects.

This addition emphasizes the tool's capacity to not only identify problems but also provide AI-driven recommendations and solutions for resolving those issues, making it even more valuable for infrastructure management and maintenance.

Checks

Provider	Checks
k8s	31
mysql	1

k8s checks

Check	Service	Category	Severity	Description
Check if a k8s PVC is in Pending state.	pvc	Health	Critical	Alerts on pending PVCs, highlighting potential delays in provisioning persistent volume claims for all the namespaces
Check if the k8s node is in Ready state.	node	Health	Critical	Ensure node health by examining readiness conditions, signaling failures if any issues are detected in the node's status
Deployment has insufficient replicas.	deployment	Health	Critical	Validate Deployments for the correct number of available replicas, highlighting any discrepancies between desired and available counts
Pod has a high restart count.	pod	Health	Critical	Identify pods for all the namespaces where certain containers have restarted more than 10 times, indicating potential instability concerns
Pod is in CrashLoopBackOff state.	pod	Health	Critical	Identify pods with containers stuck in a CrashLoopBackOff state, highlighting potential issues impacting pod stability for all the namespaces
Service has endpoints that are NotReady.	service	Health	Severe	Highlights when services have NotReady endpoints, indicating potential disruptions to service reliability for all the namespaces
Service has no endpoints.	service	Health	Severe	Identify services with no associated endpoints, highlighting potential misconfigurations impacting service connectivity
Analyzing HPAs, checking if scale targets exist and have resources	pod	HPA	High	Analyze optimal Horizontal Pod Autoscaler (HPA) configurations by ensuring associated resources (Deployments, ReplicationControllers, ReplicaSets, StatefulSets) have defined resource limits for effective auto-scaling
Check for the existence of Ingress class, service and secrets for all the namespaces	ingress	Ingress	High	Ensure proper Ingress configurations by validating associated services, secrets, and ingress classes, flagging issues if there are missing elements or misconfigured settings for all the namespaces
Check the existence of secret in Daemonset	daemonset	Daemonset, Secret	High	Ensure the presence of referenced Secrets in Daemonset volumes, reporting failures for any missing Secret within all the namespaces
Check the existence of secret in Deployment	secret	Deployment	High	Ensure the presence of referenced Secrets in Deployment volumes, reporting failures for any missing Secret for all the namespaces
Excessive Pods on Node	node	Resource Limits	High	Assesses nodes for excessive pod counts, flagging potential issues if pods near capacity thresholds based on CPU and memory resources
Find Deployments with missing configmap	configmap	Deployment	High	Ensure the presence of referenced ConfigMaps in Deployment volumes, reporting failures for any missing ConfigMap for all the namespaces
Find Pending Pods	pod	Health	High	Ensure that Pods are not in a Pending state due to scheduling issues or container creation failures, and report relevant details for diagnostics
Find Pods with missing configmap	pod	Pod, ConfigMap	High	Ensure the presence of referenced ConfigMaps in Pod containers and volumes, reporting failures for any missing ConfigMap for all the namespaces
Find Pods with missing secrets	pod	Pod, Secret	High	Ensure the presence of referenced Secrets in Pod containers, reporting failures for any missing Secret for all the namespaces
Insufficient PIDs on Node	node	Performance	High	Check if the nodes have remaining PIDs less than a set threshold
Kubernetes Node Out-of-Memory Check	node	Performance	High	Checks if any Kubernetes node is using more than 85% of its memory capacity.
Validate configmap existence in Statefulset	statefulset	StatefulSet	High	Ensure the existence of referenced ConfigMaps in StatefulSet volume claims and template volumes, reporting failures for any missing ConfigMap for all the namespaces
Validate cronjob starting deadline	cronjob	CronJob	High	Ensure CronJobs have a non-negative starting deadline, reporting failures for negative values for all the namespaces
Validate existence of configmaps in daemonsets	daemonset	DaemonSet, ConfigMap	High	Ensure the presence of referenced ConfigMaps in Daemonset volumes, reporting failures for any missing ConfigMap for all the namespaces
Verify StatefulSet has valid service	statefulset	StatefulSet	High	Verify StatefulSet's service reference, ensuring it points to an existing service in all the namespaces, reporting failures for non-existent services
Verify StatefulSet has valid storageClass	statefulset	StatefulSet	High	Validate StatefulSet's storage class, ensuring it references existing storage classes in the namespace, reporting failures for non-existent ones
Zero Scale Deployment Check	deployment	Availability	High	Verify that Deployments have a non-zero replica count, preventing unintentional scaling down to zero
Check if Kubernetes services have matching pod labels	service	Configuration	Medium	This check validates if Kubernetes service selectors match pod labels. This ensures proper routing & discovery of pods.
Pod template validation in DaemonSet	daemonset	Resource Management	Medium	Checks that the Pod template within a DaemonSet is configured correctly according to certain threshold values.
Services Target Port Match	service	Diagnostic	Medium	This check identifies service ports that do not match their target ports
Validate that network policies are in place and configured correctly	networkpolicy	Network Security	Medium	Verify Network Policy configurations, highlighting issues if policies allow traffic to all pods or if not applied to any specific pods
Zero scale detected in statefulset	statefulset	Availability	Medium	Check to ensure that no StatefulSets are scaled to zero as it might hamper availability.
Find unused DaemonSet	daemonset	DaemonSet, Cost, Resource Optimization	Low	Any DaemonSet that has been created but has no associated pods and remained unused for over 30 days.
Validate cronjobs schedule and state	cronjob	CronJob	Low	Ensure CronJobs have valid schedules and are not suspended, reporting failures for any invalid schedules or suspended jobs for all the namespaces

mysql checks

Check	Service	Category	Severity	Description
Checks max used connections	global	Connection, Thread	High	Checks max used connections reaching max count

Built With

Getting Started

Prerequisites

• Python >= 3.10
• OpenAI API Key - to have AI based functionality enabled

Installation

1. Get distibution on your machine:
   • Run pip command to install unctl from PyPI

     pip install unctl
     

2. (optional) Set OpenAI API key to be able to use --explain (-e) option

   export OPENAI_API_KEY=<your api key>
   

Kubernetes

1. (optional) Set KUBECONFIG variable to specific location other than default:

   export KUBECONFIG=<path to kube config file>
   

2. Run unctl command to see list of options:

   unctl k8s -h
   

MySQL

1. unctl is using ~/.my.cnf as config path.
2. Run unctl command to see list of options:

   unctl mysql -h
   

Config file

By default unctl is looking at ~/.config/unctl/config.yaml. Otherwise it would use default values.

To specify path to the config file use --config option:

unctl --config <path to file> {provider} ...

Sections

# section responsible for anonymisation any user data before sending it to any 3rd party service
anonymisation:
  # defines regex patterns to be used for masking any data that matches the pattern
  masks:
    - name: email
      pattern: \b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b
    - name: ip_address
      pattern: \b(?:\d{1,3}\.){3}\d{1,3}\b

Usage

unctl

% unctl -h
usage: unctl [-h] [-v] [--config CONFIG [CONFIG ...]] {k8s,mysql} ...

          Welcome to unSkript CLI Interface 

options:
  -h, --help            show this help message and exit
  -v, --version         show program`s version number and exit
  --config CONFIG [CONFIG ...]
                        Specify path to the unctl config file

unctl available providers:
  {k8s,mysql}

To see the different available options on a specific provider, run:
    unctl {provider} -h|--help

Provider

% unctl {provider} -h
usage: unctl k8s [-h] [-s] [-e] [-f] [-c CHECKS [CHECKS ...]] [--sort-by {object,check}] [--categories CATEGORIES [CATEGORIES ...]]
                 [--services SERVICES [SERVICES ...]] [-l] [--no-interactive NO_INTERACTIVE] [--list-categories] [--list-services] [-d] [-r]

options:
  -h, --help            show this help message and exit
  -s, --scan            Run a provider scan
  -e, --explain         Explain failures using AI
  -f, --failing-only    Show only failing checks
  -c CHECKS [CHECKS ...], --checks CHECKS [CHECKS ...]
                        Filter checks by IDs
  --sort-by {object,check}
                        Sort results by 'object' (default) or 'check'
  --categories CATEGORIES [CATEGORIES ...]
                        Filter checks by category
  --services SERVICES [SERVICES ...]
                        Filter checks by services
  -l, --list-checks     List available checks
  --no-interactive NO_INTERACTIVE
                        Interactive mode is not allowed. Prompts will be skipped
  --list-categories     List available categories
  --list-services       List available services
  -d, --diagnose        Run fixed diagnosis
  -r, --remediate       Create remediation plan

Roadmap

• K8s checks - in progress
• MySQL checks - in progress
• Elastic Search checks
• AWS checks
• GCP checks

Contact

Abhishek Saxena: [e-mail]

Official website: https://unskript.com/