Pets not cattle. A straightforward way to manage configuration files.
Project description
Hamstercage - Config Management for Pets, not Cattle
Overview
If you work professionally with many machines and config, you like will have heard "cattle, not pets" as the philosophy for managing machines, VMs, etc., and you're likely using Ansible, Puppet, SaltStack, or another configuration management system that allows you to express configuration as code. This approach works well when you have many targets that share many traits, you have a lab where you can test configuration changes, and you have full time staff to take care of it all.
However, if you're running a handful of boxes or VPSes for a small organisation, or just for yourself and your friends and family, your workflow might actually look quite different: you make changes to the live configuration of your web server, for example, and after you're satisfied that everything is working, you might want to save the key bits of config somewhere safe, so you can refer back to it later. Setting up any of the heavy tools can be cumbersome, especially for making quick provisional changes: in the worstcase scenario, you modify a file in the source repo, commit it, then run the tool to apply it to your machine.
Hamstercage aims to make it easy to save and restore your config by using a Git repo, by editing the config files directly on the target machine, then saving the new config into the repository. In other words: pets, not cattle.
Hamstercage is geared towards managing config files as complete files. To keep things simple, there are no facilities to update individual lines in files, update system configuration settings through some API, or other more complex logic. Hamstercage can be used to manage shell script files or binaries for custom tools, however.
To allow one repository to be used for multiple targets, sets of files can be managed. Each set is called a tag. You can select the tags to use each time you run Hamstercage. The manifest also contains a list of hostnames and the tags to use for each. This makes it possible to run the same Hamstercage command on multiple hosts, and have files be applied to each according to their respective purpose.
Theory of Operation
Hamstercage keeps a list of files under management in a manifest file, by default hamstercage.yaml
. The various Hamstercage commands can be used to add, update, and remove files from the manifest, but you can always make changes manually, for example to correct the owner or mode of a file.
Managed files are kept not as a single list, but as multiple lists, one for each tag. A tag is simply a label for a set of files. You are free to choose tags to organize sets of files as you see fit, for example, by purpose, application, operating system, or any other metric.
In addition to the managed files, the manifest also contains a list of hostnames, with the tags that should be applied to each hostname. This allows you to run the same Hamstercage command across multiple hosts and have files applied according to the hosts' respective tags.
The contents of files is kept alongside the manifest, using the path of the target. For example, a target file /etc/bash_profile
would be stored in repo/all/etc/bash_profile
.
While Hamstercage manages files very efficiently, there are some configuration tasks that require manipulation of existing files, refreshing a set of files, or restarting a daemon. Hamstercage does not attempt to implement any such actions. Hooks allow you to supply your own custom logic for these kinds of actions. You can define hooks to run before or after a command is executed (for example before a save
or after an apply
). Hooks are defined on tags, so you can cater the commands to the set of files that are managed. Hooks can be individual shell commands, any executable including scripts, or Python programs that are executed inside Hamstercages Python process.
Installation
Hamstercage is a Python module. You can install it from this source repository, or from PyPi.
pip install hamstercage
Using Hamstercage
After installation, Hamstercage is available on the path as hamstercage
.
Creating a new Hamstercage repo
Hamstercage needs a manifest file in order to track files managed through it. You can create this file manually, or you can have Hamstercage create a template file for you:
hamstercage init
This creates hamstercage.yaml
in the current directory.
The Hamstercage Manifest
All information about file entries is stored in the manifest file hamstercage.yaml
. This YAML file consists of these top-level entries:
hosts
: each entry describes one host, in particular the set of tags that apply to ittags
: each entry contains a list of files managed for this tag
This is a very minimal example with one host, one tag and one file:
hosts:
testing.example.com:
description: ''
tags:
- all
tags:
all:
description: files that apply to all hosts
entries:
foo.txt:
group: staff
mode: 0o644
owner: stb
type: file
Hosts
Each entry in this dict defines one host. Hosts are distinguished by their fully-qualified hostname. When running Hamstercage, you can override the hostname by specifying the -h
option. Each host definition can have these fields:
description
: allows you to add a description or note to this entry. Optional.tags
: a list of one or more tags to apply to this host.
Tags
Each entry in this dict defines one tag. Tag names can be freely chosen; it is recommended to stick to alpha-numeric identifiers though, to keep the names compatible with shell scripts, etc. Each entry has these fields:
description
: allows you to add a description or note to this entry. Optional.entries
: a dict of files managed through this tag. See below for details.hooks
: a dict of hook scripts that will be run. See below for details.
Tags: Entries
The entries field in a tag entry describes files, directories, and symbolic links that are managed through that tag. Each entry can have these fields:
group
: the group name owning this file.mode
: the access mode for this file. You need to specify the mode as an octal number (0o644
) or as a string in quotes ("644"
). If you simply give a number, it will be interpreted as a base-10 integer, leading to unexpected mode bitsowner
: the user name owning this file.target
: the path the link points to, only applicable totype
=link
type
: must be one ofdir
,file
, orlink
.
Note that Hamstercage only manages symbolic links (soft links). Hard links are not supported.
Tags: Hooks
The hooks
dict in a tag allows you to hook scripts into Hamstercages execution. The name of each entry defines when to run the script; the contents of the dict define what command to execute.
The name consists of two parts separated by a dash: the prefix pre
or post
, and the name of the Hamstercage command. For example, the hook defined by the name pre-save
will be executed when you run hamstercage save
, just before the contents of the files will be copied from the target system to the repository. A hook post-apply
will be run right after all files have been updated by the command hamstercage apply
.
Hamstercage searches for a matching hook definition in this order:
- for the exact match step
-
command (for example,pre-save
) - for the wildcard match
*-
command (for example,*-save
) - for the wildcard match step
-*
(for example,pre-*
) - for the fallback wildcard
*
.
The first match found will be executed.
Each entry has these fields:
command
: the command or script to execute.description
: allows you to add a description or note to this entry. Optional.type
: the type of command or script to run.exec
: run the program specified bycommand
. Ifcommand
is not an absolute path, the system will search for it on thePATH
. See subprocess.Popen for details. The command will be invoked with the Hamstercage command, the step, and the tag the hook is defined in as parameters, for example.../myhook.sh apply post all
.python
:command
should specify a Python script. A relative path is interpreted relative to the repository directory. See below for global variables available to the script.shell
: runcommand
through the shell. Typically, this will be the shell of the user running Hamstercage.
The exec
and shell
hooks receive the following environment variables:
HAMSTERCAGE_CMD
: the command being executedHAMSTERCAGE_MANIFEST
: the manifest file path nameHAMSTERCAGE_HOOK
: the name of the hook entryHAMSTERCAGE_REPO
: the repo directory, which is the directory the manifest file lives inHAMSTERCAGE_STEP
: the step (pre
orpost
)HAMSTERCAGE_TAG
: the tag name this hook is defined in
The python
script receives these global variables:
cmd
: the command being executedmanifest
: the manifest objecthook
: the name of the hook entryrepo
: the repo directory, which is the directory the manifest file lives instep
: the step (pre
orpost
)tag
: the tag object this hook is defined in__file__
: the Python hook file__name__
: the constant__hamstercage__
.
Command Reference
Hamstercage commands take a number of options and parameters.
General Options
Target Directory -d
/ --directory
The directory where the managed files live, default /
. You can set this to a different path when restoring config from a backup, or when managing a jail or container from the outside.
Hostname -n
/ --hostname
The hostname of the local host. By default the value of hostname(1)
. When applying files to the target directory, the hostname is used to obtain the default list of tags to use. Specifying this can be useful when restoring a host from backup, or when working on a container or jail.
Manifest File -f
/ --file
The name of the manifest file that tracks all managed files, as well as settings for the repository. Default is ./hamstercage.yaml
.
Repository Base Directory -r
/ --rep
All managed files are stored under this directory. The default is the same directory the manifest file lives in.
Tags -t
/ --tag
When adding files or applying them to the target directory, one or more tags can be specified. See the add
and apply
commands.
Verbose output -v
/ --verbose
By default, Hamstercage will only print warnings or error. With this flag, progress will be reported on all operations.
Commands
Adding Files add
hamstercage -t mytag add file...
Add one or more files to the repository. You need to specify exactly one tag, which determines the tag the file will be added to. The file will be added to the manifest, and the contents of the file copied to the repository.
Applying Files apply
hamstercage apply [file...]
Apply files from the repository to the target. Without any further options and parameters, will apply all files from the default tags for this hostname. Specify one or more tags to limit files to those in those tags, or specify filenames that should be applied.
Adding a file that is already in the manifest for the given tag is an error.
Determining Differences diff
hamstercage diff [file...]
Prints out a unified diff between all files in the repository and in the target. This allows you to preview what changed apply
or save
will make.
Initialize a New Repository init
hamstercage init
Creates a new manifest file, by default ./hamstercage.yaml
. Combine this with git init
to create a new Hamstercage repository.
List Entries list
hamstercage list [-l]
Lists all entries in the manifest. When -l
is given, will print more details on each entry.
Remove Files From Repository remove
hamstercage -t tag remove file...
Removes one or more files from the specified tag. Does not change files in the target.
Save Target to Repository save
hamstercage save [file...]
Updates the manifest entries and repository files from the target.
Adding Tags tag add
Adds a new tag to the manifest.
hamstercage tag add [-d description] [file...]
Developing Hamstercage
Poetry For Dependency Management and Building
The project uses Poetry, which you should install locally. After installing Poetry, you can install all necessary dependencies:
poetry install
Source Code Formatting With Black
The GitHub workflow checks source code formatting with black.
To format all code automatically:
poetry run black .
When working on the code, you might want to configure your IDE to automatically reformat the code with black.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for hamstercage-0.0.16-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 59d2dbd1dfbbffbc757aebcec5c6a2546851fac6136946f4ea6b17e45e5223dc |
|
MD5 | c26011ab92979062d2d68cdf053ba869 |
|
BLAKE2b-256 | ee73a9a65759d362408acc499df498b9f78999e25ecfd518e47a052ade70239f |