Skip to main content

Percussive maintenance.

Collection of various tools to keep things ship-shape. Not particularly bright tools, but good for a first-pass.

  1. Neutron resource reaper

neutron-reaper {info, delete} {ip, port} <grace-days> [ --dbversion ocata ]

Reclaims idle floating IPs and cleans up stale ports.

  1. Conflicting Ironic/Neutron MACs

conflict-macs {info, delete} ( --ignore-from-ironic-config <path to ironic.conf> | --ignore-subnet <subnet UUID> )

The Ironic subnet must be provided—directly via ID or determined from a config—otherwise the script would think that they are in conflict.

  1. Undead Instances clinging to nodes

undead-instances {info, delete}

Nova instances that have been put to rest but still cling to Ironic nodes, preventing the next generation from being…ensouled? Checks for the inconsistency and fixed it.

  1. Clean up after IPMI errors

ironic-error-resetter {info, reset}

Resets Ironic nodes in error state with a known, common error. Records those resets on the node metadata (extra field) and refuses after a magic number of attempts.

  1. Update orphaned resource providers

orphan-resource-providers {info, update}

Detects and updates resource providers whose UUID has not been updated to match a recreated Nova compute node.

Common options:

  • --slack <json-options> - if provided, used to post notifications to Slack

  • --osrc <rc-file> - alternate way to feed in the OS authentication vars

Setup/Config

  1. Get code

mkdir -p /root/scripts/hammers
cd /root/scripts/hammers
git clone https://github.com/ChameleonCloud/hammers.git
  1. Create environment

virtualenv /root/scripts/hammers/venv
/root/scripts/hammers/venv/bin/pip install -r /root/scripts/hammers/hammers/requirements.txt
/root/scripts/hammers/venv/bin/pip install -e /root/scripts/hammers/hammers
  1. Set up credentials for OpenStack and Slack

The below cronjob assumes the OS var file is at /root/adminrc and the Slack vars are in /root/scripts/slack.json. The Slack file is a JSON with a root key "webhook" that is a URL to post to (keep secret!) and another root key "hostname_name" that is a mapping of FQDNs to pretty names. Example:

{
    "webhook": "https://hooks.slack.com/services/...super-seekrit...",
    "hostname_names": {
        "m01-07.chameleon.tacc.utexas.edu": "CHI@TACC",
        "m01-03.chameleon.tacc.utexas.edu": "KVM@TACC",
        "admin01.uc.chameleoncloud.org": "CHI@UC"
    }
}
  1. Add cronjob(s) to Puppet:

$slack_json_loc = '/root/scripts/slack.json'
$osrc_loc = '/root/adminrc'
$venv_bin = '/root/scripts/hammers/venv/bin'

cron { 'hammers-neutronreaper-ip':
  command => "$venv_bin/neutron-reaper delete ip 14 --dbversion ocata --slack $slack_json_loc --osrc $osrc_loc [--kvm if at KVM site] 2>&1 | /usr/bin/logger -t hammers-neutronreaper-ip",
  user => 'root',
  hour => 5,
  minute => 20,
}
cron { 'hammers-ironicerrorresetter':
  command => "$venv_bin/ironic-error-resetter info --slack $slack_json_loc --osrc $osrc_loc 2>&1 | /usr/bin/logger -t hammers-ironicerrorresetter",
  user => 'root',
  hour => 5,
  minute => 25,
}
cron { 'hammers-conflictmacs':
  command => "$venv_bin/conflict-macs info --slack $slack_json_loc --osrc $osrc_loc --ignore-from-ironic-conf /etc/ironic/ironic.conf 2>&1 | /usr/bin/logger -t hammers-conflictmacs",
  user => 'root',
  hour => 5,
  minute => 30,
}
cron { 'hammers-undeadinstances':
  command => "$venv_bin/undead-instances info --slack $slack_json_loc --osrc $osrc_loc 2>&1 | /usr/bin/logger -t hammers-undeadinstances",
  user => 'root',
  hour => 5,
  minute => 35,
}
cron { 'hammers-orphanresourceproviders':
  command => "$venv_bin/orphan-resource-providers info --slack $slack_json_loc 2>&1 | /usr/bin/logger -t hammers-orphanresourceproviders",
  user => 'root',
  hour => 5,
  minute => 40,
}

Metadata

Release files for hammers 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for hammers 0.2.0
File Size Uploaded
hammers-0.2.0.tar.gz 53.1 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for hammers 0.2.0
File Interpreter ABI Platform
hammers-0.2.0-py2.py3-none-any.whl Python 3, Python 2 none any Details

Total release size: 121.7 kB

Release files / hammers-0.2.0.tar.gz

Download URL hammers-0.2.0.tar.gz
Size 53.1 kB
Tags Source
SHA-256 checksum
How to use checksums
b9a540abc8ff134fd277ebb8ae34ed24d66672467b4edb55a039e8c3f98975a2
BLAKE2b-256 checksum
How to use checksums
f89a8f3112d3b73a89e07bd69695993eff4bdaaa5d59048550cda452e46b001c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.11.0 pkginfo/1.4.2 requests/2.18.4 setuptools/39.2.0 requests-toolbelt/0.8.0 tqdm/4.23.3 CPython/2.7.15

Release files / hammers-0.2.0-py2.py3-none-any.whl

Download URL hammers-0.2.0-py2.py3-none-any.whl
Size 68.5 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
281708f1a830886eed3de8e538d92ca48a06ab30d34987253af21ac7c87bbfd9
BLAKE2b-256 checksum
How to use checksums
99133e95f17440c3db07bc0bca7c01c82ef71c3cb2063d0dc0a048b43464c151
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.11.0 pkginfo/1.4.2 requests/2.18.4 setuptools/39.2.0 requests-toolbelt/0.8.0 tqdm/4.23.3 CPython/2.7.15

Release history Release notifications | RSS feed

This release

0.2.0 This release

2 release files

0.1.12

2 release files

0.1.9

2 release files

0.1.8

2 release files

0.1.7

2 release files

0.1.6

2 release files

0.1.5

2 release files

0.1.4

2 release files

0.1.3

2 release files

0.1.2

2 release files

0.1.1

2 release files

0.1.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page