yog 1.3.4

The Gate and Key

Yog

Overview

An opinionated docker-and-ssh-centric declarative system management tool.

pip3 install yog

Features:

• Like puppet or ansible but a lot smaller and focused on docker, files, and cron
• agentless - runs entirely on top of ssh
• entirely defers auth(z/n) to ssh and the remote system's user permissions

Command summary:

• yog: Applies configurations to hosts. e.g. yog myhost.mytld applies the config from ./domains/mytld/myhost.yml.
• yog-repo: Manages a docker repository. yog-repo push uses the contents of ./yog-repo.conf to build an image and push it to the configured registry with the configured name and tag.

Example run:

$ yog myhost.example.com
[2022-12-26 11:01:52,514] [INFO]: [myhost.example.com]
[2022-12-26 11:01:59,121] [INFO]: [myhost.example.com][files]: OK [hello_world.html]
[2022-12-26 11:01:59,274] [INFO]: [myhost.example.com][files]: OK [helloworld-nginx.conf]
[2022-12-26 11:02:07,117] [INFO]: [myhost.example.com][docker]: OK registry@sha256:8be26f81ffea54106bae012c6f349df70f4d5e7e2ec01b143c46e2c03b9e551d

Setup

1. Configure docker to listen on localhost's port 4243 (which is the default). See below.
2. Use ssh-copy-id to copy your ssh key to all the servers you wish to manage. You can look into ssh certificates if you want a more general ssh PKI solution.
3. Configure the target system to allow you to use sudo without a password. [2]
4. That's it. You now serve the nameless mist. Do you hear whippoorwills?

Docker port listening setup (step 1)

ssh myhost
sudo systemctl edit docker

And add -H tcp://127.0.0.1:4243 to the command. So for me, that file looks like:

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// -H tcp://127.0.0.1:4243

Yog will apply files before docker containers, so you might also want to add a yog file entry like so:

files:
  - src: docker-override.conf
    dest: /etc/systemd/system/docker.service.d/override.conf
    root: yes
    hupcmd: sudo systemctl restart docker

Usage

Yog uses YML files that it calls "necronomicons" for configuration of hosts. It's organized hierarchically so that a necronomicon for "mytld" will be applied to all hosts under that TLD.

Let's learn by example:

Suppose we have a folder, that can be named whatever we want, at ~/projects/my-config. This is the root of a git repo, and is also the root of our yog configuration. Make this your current working dir, or pass --root-dir.

$ cd ~/projects/my-config

.
├── domains
│      ├── com
│      │      └── example
│      │          └── myhost.yml
│      └── com.yml
└── files
    ├── example.txt
    ├── hello_world.html
    └── helloworld-nginx.conf

4 directories, 5 files

Files that can be sent to hosts are stored under files.

Host configurations - necronomicons - are stored under domains.

If we want to apply myhost.yml, we run:

yog myhost.example.com

Example output:

$ yog myhost.example.com
[2022-12-26 11:01:52,514] [INFO]: [myhost.example.com]
[2022-12-26 11:01:59,121] [INFO]: [myhost.example.com][files]: OK [hello_world.html]
[2022-12-26 11:01:59,274] [INFO]: [myhost.example.com][files]: OK [helloworld-nginx.conf]
[2022-12-26 11:02:07,117] [INFO]: [myhost.example.com][docker]: OK registry@sha256:8be26f81ffea54106bae012c6f349df70f4d5e7e2ec01b143c46e2c03b9e551d

Necronomicon format

Let's look at a necronomicon.

files:
  - src: hello_world.html
    dest: /srv/hello_world/hello_world.html
    root: yes
  - src: helloworld-nginx.conf
    dest: /etc/nginx/conf.d/helloworld.conf
    root: yes
    hupcmd: sudo systemctl restart nginx


docker:
  - image: registry
    name: my-registry
    fingerprint: sha256:8be26f81ffea54106bae012c6f349df70f4d5e7e2ec01b143c46e2c03b9e551d
    volumes:
      images: /var/lib/registry
    ports:
      5000: 5000
    env:
      REGISTRY_STORAGE_DELETE_ENABLED: true

Files

Files are checked for equality via hash-comparison. I've found this a useful way to manage:

• cron files in /etc/cron.d/
• Root certificates to put in the system trust store[1]
• random config files

Attributes:

• src: the source. This is a relative path rooted at the files directory in the hierarchy. You can use intermediate dirs.
• dest: the destination filepath on the managed host. This is an absolute path.
• root: whether to sudo to root for the file put. This mainly picks who owns the file + can access files, but this might have other useful properties for your use case. If set to no, the put operation is run as your ssh user.
• hupcmd: a command to run after the file is placed. A common thing in ye olde days was to send SIGHUP to a process which would handle it by reloading the config. Commonly nowadays you might be using hupcmd: sudo systemctl reload nginx

Docker containers

Docker containers are compared on all specified attributes and won't unnecessarily restart containers.

Attributes:

• image: the docker repository name. e.g. itzg/minecraft-server or dockerrepo.local:5000/mything
• name: the container name.
• fingerprint: sha digest of the desired version. Tags are bad news bears so we don't support them. This is called fingerprint instead of digest because I didn't know they were called digests when I first coded this and then never changed it once I did.
• volumes: volumes to attach. see below.
• ports: ports to open. see below.
• env: environment variables to set.

Volumes

For volumes, the key is the volume name and the value is the mount point.

For bind mounts, the key is the host path and the value is the container path.

Ports

The key is the host addr/port, and the value is the dest container port. Examples:

192.168.1.103:53/tcp: 53/tcp
192.168.1.103:53/udp: 53/udp
127.0.0.1:53/tcp: 53/tcp
127.0.0.1:53/udp: 53/udp
33200: 33200
8080: 3000 # host port 8080 maps to container port 3000

Footnotes

[1] This is one of those things where I feel like you probably shouldn't manage root certs like this but I have yet to regret it? It's not a cryptographic secret, so.

[2] Also something that people say you probably shouldn't do but I've yet to regret. If your user is in the docker group it's basically root anyway from a threat modeling perspective.

yog-repo

Yog also includes a tool for pushing images to your local docker registry. I haven't documented it yet, apologies.