Skip to main content

Chaos for Clouds.

Project description

license python versions status pypi version PyPI - Downloads

Chaotic - Chaos for Clouds

Chaotic evaluates a plan, how it will bring chaos in your Cloud environment.

Depending on the Cloud API used, it may kill allocations (Hashicorp Nomad), reboot or stop/start virtual machines in your Cloud environment.

With no arguments given, Chaotic runs as a "one shot" meant to be executed as cron job. Passing --periodic runs it as daemon with configurable mintute intervals --interval 5 (1 is the default).

Note: The config is re-read on every interval, no need to restart the service after a config change.

Clouds

Currently implemented Clouds:

  • DigitalOcean
  • Vultr
  • Hetzner Cloud
  • Proxmox KVM
  • CloudStack
  • Hashicorp Nomad
  • Exoscale
  • cloudscale.ch

Install

pip3 install -U chaotic-ngine

Configure

Create a file named config.yaml or use the env var CHAOTIC_CONFIG to point to a config file (also see the example directory):

export CHAOTIC_CONFIG=config_nomad.yaml

Exclude times

Define times when the bot should not doing real actions (it will run in dry-run):

---
kind: ...
excludes:
  weekdays:
    - Sun
    - Sat
  times_of_day:
    - 22:00-08:00
    - 11:00-14:00
  days_of_year:
    - Jan01
    - Apr01
    - May01
    - Aug01
    - Dec24

Exoscale

Chaotic will stop a server selected by an optional filter tag and stop/start it with a delay of a configurable time (default 60s).

export EXOSCALE_API_KEY="..."
export EXOSCALE_API_SECRET="..."
---
kind: exoscale
dry_run: false
configs:

  # Optional, filter tag
  tag:
    key: chaos
    value: enabled

  # Optional, 60 seconds is the default
  wait_before_restart: 60

CloudStack

Chaotic will stop a server selected by an optional filter tag and stop/start it with a delay of a configurable time (default 60s).

export CLOUDSTACK_API_KEY="..."
export CLOUDSTACK_API_SECRET="..."
export CLOUDSTACK_API_ENDPOINT="..."
---
kind: cloudstack
dry_run: false
configs:

  # Optional, filter tag
  tag:
    key: chaos
    value: enabled

  # Optional, 60 seconds is the default
  wait_before_restart: 60

Vultr

Chaotic will stop a server selected by an optional filter tag and stop/start it with a delay of a configurable time (default 60s).

export VULTR_API_KEY="..."
---
kind: vultr
dry_run: true
configs:

  # Optional instance tag filter
  tag: "chaos=opt-in"

  # Optional, 60 seconds is the default
  wait_before_restart: 60

Cloudscale.ch

Chaotic will stop a server selected by an optional filter tag and stop/start it with a delay of a configurable time (default 60s).

Config

export CLOUDSCALE_API_TOKEN="..."
---
kind: cloudscale_ch
dry_run: true
configs:

  # Optional server tag filter
  filter_tag: "chaos=opt-in"

  # Optional, 60 seconds is the default
  wait_before_restart: 60

Hetzner Cloud

Chaotic will stop a server selected by an optional filter label and stop/start it with a delay of a configurable time (default 60s).

Config

export HCLOUD_API_TOKEN=...
---
kind: hcloud
dry_run: false
configs:

  # Optional server label filter
  label: "chaos=enabled"

  # Optional, 60 seconds is the default
  wait_before_restart: 60

DigitalOcean Cloud

Chaotic will stop a droplet selected by an optional filter tag and stop/start it with a delay of a configurable time (default 60s).

Config

export DIGITALOCEAN_ACCESS_TOKEN=...
---
kind: digitalocean
dry_run: false
configs:

  # Optional droplet tag filter
  tag: "chaos:enabled"

  # Optional, 60 seconds is the default
  wait_before_restart: 60

Nomad

Chaotic will send an allocation signal to an allocation in the available namespaces selected by an allow list.

Config

export NOMAD_ADDR=http://nomad.example.com:4646
---
kind: nomad
dry_run: true
configs:

  # Signals to choose from
  signals:
    - SIGKILL

  # Optional: namespace allowlist
  namespace_allowlist:
    - example-prod
    - foobar-prod

  # Optional: namespace denylist
  namespace_denylist:
    - default

Proxmox KVM

Chaotic will stop a VM stop/start it with a delay of a configurable time (default 60s).

export PROXMOX_API_HOST="pve1.example.com"
export PROXMOX_API_USER="root@pam"
export PROXMOX_API_PASSWORD="..."
---
kind: proxmox_kvm
dry_run: false
configs:

  # Optional: Do not shutdown VMs having a lower uptime in minutes
  min_uptime: 60

  # Optional: Do not shutdown VMs in this name list
  denylist:
    - my-single-vm

  # Optional: 60 seconds is the default
  wait_before_restart: 60

Run

chaos-ngine

What you should see (cloudscale.ch):

2021-06-09 09:01:25,433 - cloudscale.log:INFO:Started, version: 0.6.2
2021-06-09 09:01:25,433 - cloudscale.log:INFO:Using profile default
2021-06-09 09:01:25,433 - cloudscale.log:INFO:API Token used: xyz...
2021-06-09 09:01:25,433 - chatic:INFO:Querying with filter_tag: None
2021-06-09 09:01:25,433 - cloudscale.log:INFO:HTTP GET to https://api.cloudscale.ch/v1/servers
2021-06-09 09:01:25,651 - cloudscale.log:INFO:HTTP status code 200
2021-06-09 09:01:25,652 - chatic:INFO:Choose server app3
2021-06-09 09:01:25,653 - chatic:INFO:Stopping server app3
2021-06-09 09:01:25,653 - cloudscale.log:INFO:HTTP POST to https://api.cloudscale.ch/v1/servers/d5628484-a6eb-4ea9-b3ef-ba8da2bb9fe0/stop
2021-06-09 09:01:26,336 - cloudscale.log:INFO:HTTP status code 204
2021-06-09 09:01:26,336 - chatic:INFO:Sleeping for server 60
2021-06-09 09:02:26,393 - cloudscale.log:INFO:HTTP POST to https://api.cloudscale.ch/v1/servers/d5628484-a6eb-4ea9-b3ef-ba8da2bb9fe0/start
2021-06-09 09:02:26,955 - cloudscale.log:INFO:HTTP status code 204
2021-06-09 09:02:26,956 - chatic:INFO:done

Docker / Nomad run

See https://gitlab.com/ngine/docker-images/chaotic

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

chaotic-ngine-0.7.1.tar.gz (11.1 kB view hashes)

Uploaded source

Built Distribution

chaotic_ngine-0.7.1-py3-none-any.whl (13.5 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page