A health check and RCA tool for kubernetes
Project description
unctl
Table of Contents
-
About The Project
- Built With
- Checks
-
Getting Started
- Prerequisites
- Installation
- Config file
- Usage
- Roadmap
- Contact
About The Project
unctl
is a versatile command-line tool designed to perform a wide range of checks and inspections on various components of your infrastructure. It provides a unified interface to assess the health and performance of different services and platforms, and goes beyond mere diagnosis. With built-in AI capabilities, it guides you seamlessly from system diagnostic to remediation, offering intelligent solutions to address any issues it detects.
This addition emphasizes the tool's capacity to not only identify problems but also provide AI-driven recommendations and solutions for resolving those issues, making it even more valuable for infrastructure management and maintenance.
Checks
Provider | Checks |
---|---|
k8s | 31 |
mysql | 1 |
k8s checks
Check | Service | Category | Severity | Description |
---|---|---|---|---|
Check if a k8s PVC is in Pending state. | pvc | Health | Critical | Alerts on pending PVCs, highlighting potential delays in provisioning persistent volume claims for all the namespaces |
Check if the k8s node is in Ready state. | node | Health | Critical | Ensure node health by examining readiness conditions, signaling failures if any issues are detected in the node's status |
Deployment has insufficient replicas. | deployment | Health | Critical | Validate Deployments for the correct number of available replicas, highlighting any discrepancies between desired and available counts |
Pod has a high restart count. | pod | Health | Critical | Identify pods for all the namespaces where certain containers have restarted more than 10 times, indicating potential instability concerns |
Pod is in CrashLoopBackOff state. | pod | Health | Critical | Identify pods with containers stuck in a CrashLoopBackOff state, highlighting potential issues impacting pod stability for all the namespaces |
Service has endpoints that are NotReady. | service | Health | Severe | Highlights when services have NotReady endpoints, indicating potential disruptions to service reliability for all the namespaces |
Service has no endpoints. | service | Health | Severe | Identify services with no associated endpoints, highlighting potential misconfigurations impacting service connectivity |
Analyzing HPAs, checking if scale targets exist and have resources | pod | HPA | High | Analyze optimal Horizontal Pod Autoscaler (HPA) configurations by ensuring associated resources (Deployments, ReplicationControllers, ReplicaSets, StatefulSets) have defined resource limits for effective auto-scaling |
Check for the existence of Ingress class, service and secrets for all the namespaces | ingress | Ingress | High | Ensure proper Ingress configurations by validating associated services, secrets, and ingress classes, flagging issues if there are missing elements or misconfigured settings for all the namespaces |
Check the existence of secret in Daemonset | daemonset | Daemonset, Secret | High | Ensure the presence of referenced Secrets in Daemonset volumes, reporting failures for any missing Secret within all the namespaces |
Check the existence of secret in Deployment | secret | Deployment | High | Ensure the presence of referenced Secrets in Deployment volumes, reporting failures for any missing Secret for all the namespaces |
Excessive Pods on Node | node | Resource Limits | High | Assesses nodes for excessive pod counts, flagging potential issues if pods near capacity thresholds based on CPU and memory resources |
Find Deployments with missing configmap | configmap | Deployment | High | Ensure the presence of referenced ConfigMaps in Deployment volumes, reporting failures for any missing ConfigMap for all the namespaces |
Find Pending Pods | pod | Health | High | Ensure that Pods are not in a Pending state due to scheduling issues or container creation failures, and report relevant details for diagnostics |
Find Pods with missing configmap | pod | Pod, ConfigMap | High | Ensure the presence of referenced ConfigMaps in Pod containers and volumes, reporting failures for any missing ConfigMap for all the namespaces |
Find Pods with missing secrets | pod | Pod, Secret | High | Ensure the presence of referenced Secrets in Pod containers, reporting failures for any missing Secret for all the namespaces |
Insufficient PIDs on Node | node | Performance | High | Check if the nodes have remaining PIDs less than a set threshold |
Kubernetes Node Out-of-Memory Check | node | Performance | High | Checks if any Kubernetes node is using more than 85% of its memory capacity. |
Validate configmap existence in Statefulset | statefulset | StatefulSet | High | Ensure the existence of referenced ConfigMaps in StatefulSet volume claims and template volumes, reporting failures for any missing ConfigMap for all the namespaces |
Validate cronjob starting deadline | cronjob | CronJob | High | Ensure CronJobs have a non-negative starting deadline, reporting failures for negative values for all the namespaces |
Validate existence of configmaps in daemonsets | daemonset | DaemonSet, ConfigMap | High | Ensure the presence of referenced ConfigMaps in Daemonset volumes, reporting failures for any missing ConfigMap for all the namespaces |
Verify StatefulSet has valid service | statefulset | StatefulSet | High | Verify StatefulSet's service reference, ensuring it points to an existing service in all the namespaces, reporting failures for non-existent services |
Verify StatefulSet has valid storageClass | statefulset | StatefulSet | High | Validate StatefulSet's storage class, ensuring it references existing storage classes in the namespace, reporting failures for non-existent ones |
Zero Scale Deployment Check | deployment | Availability | High | Verify that Deployments have a non-zero replica count, preventing unintentional scaling down to zero |
Check if Kubernetes services have matching pod labels | service | Configuration | Medium | This check validates if Kubernetes service selectors match pod labels. This ensures proper routing & discovery of pods. |
Pod template validation in DaemonSet | daemonset | Resource Management | Medium | Checks that the Pod template within a DaemonSet is configured correctly according to certain threshold values. |
Services Target Port Match | service | Diagnostic | Medium | This check identifies service ports that do not match their target ports |
Validate that network policies are in place and configured correctly | networkpolicy | Network Security | Medium | Verify Network Policy configurations, highlighting issues if policies allow traffic to all pods or if not applied to any specific pods |
Zero scale detected in statefulset | statefulset | Availability | Medium | Check to ensure that no StatefulSets are scaled to zero as it might hamper availability. |
Find unused DaemonSet | daemonset | DaemonSet, Cost, Resource Optimization | Low | Any DaemonSet that has been created but has no associated pods and remained unused for over 30 days. |
Validate cronjobs schedule and state | cronjob | CronJob | Low | Ensure CronJobs have valid schedules and are not suspended, reporting failures for any invalid schedules or suspended jobs for all the namespaces |
mysql checks
Check | Service | Category | Severity | Description |
---|---|---|---|---|
Checks max used connections | global | Connection, Thread | High | Checks max used connections reaching max count |
Built With
Getting Started
Prerequisites
- Python >= 3.10
- OpenAI API Key - to have AI based functionality enabled
Installation
- Get distibution on your machine:
- Run
pip
command to installunctl
from PyPIpip install unctl
- Run
- (optional) Set OpenAI API key to be able to use
--explain (-e)
optionexport OPENAI_API_KEY=<your api key>
Kubernetes
- (optional) Set
KUBECONFIG
variable to specific location other than default:export KUBECONFIG=<path to kube config file>
- Run unctl command to see list of options:
unctl k8s -h
MySQL
unctl
is using~/.my.cnf
as config path.- Run unctl command to see list of options:
unctl mysql -h
Config file
By default unctl
is looking at ~/.config/unctl/config.yaml
. Otherwise it would use default values.
To specify path to the config file use --config
option:
unctl --config <path to file> {provider} ...
Sections
# section responsible for anonymisation any user data before sending it to any 3rd party service
anonymisation:
# defines regex patterns to be used for masking any data that matches the pattern
masks:
- name: email
pattern: \b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b
- name: ip_address
pattern: \b(?:\d{1,3}\.){3}\d{1,3}\b
Usage
unctl
% unctl -h
usage: unctl [-h] [-v] [--config CONFIG [CONFIG ...]] {k8s,mysql} ...
Welcome to unSkript CLI Interface
options:
-h, --help show this help message and exit
-v, --version show program`s version number and exit
--config CONFIG [CONFIG ...]
Specify path to the unctl config file
unctl available providers:
{k8s,mysql}
To see the different available options on a specific provider, run:
unctl {provider} -h|--help
Provider
% unctl {provider} -h
usage: unctl k8s [-h] [-s] [-e] [-f] [-c CHECKS [CHECKS ...]] [--sort-by {object,check}] [--categories CATEGORIES [CATEGORIES ...]]
[--services SERVICES [SERVICES ...]] [-l] [--no-interactive NO_INTERACTIVE] [--list-categories] [--list-services] [-d] [-r]
options:
-h, --help show this help message and exit
-s, --scan Run a provider scan
-e, --explain Explain failures using AI
-f, --failing-only Show only failing checks
-c CHECKS [CHECKS ...], --checks CHECKS [CHECKS ...]
Filter checks by IDs
--sort-by {object,check}
Sort results by 'object' (default) or 'check'
--categories CATEGORIES [CATEGORIES ...]
Filter checks by category
--services SERVICES [SERVICES ...]
Filter checks by services
-l, --list-checks List available checks
--no-interactive NO_INTERACTIVE
Interactive mode is not allowed. Prompts will be skipped
--list-categories List available categories
--list-services List available services
-d, --diagnose Run fixed diagnosis
-r, --remediate Create remediation plan
Roadmap
- K8s checks - in progress
- MySQL checks - in progress
- Elastic Search checks
- AWS checks
- GCP checks
Contact
Abhishek Saxena: abhishek@unskript.com
Official website: https://unskript.com/
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.