distrobuilder-menu

• 

• A python console frontend to Distrobuilder for building standard or customised LXD / LXC images

• Main Menus (LXD | LXC)

• Install from pypi.org

  • pipx install distrobuilder-menu

• Install Arch Linux package from AUR

  • yay distrobuilder-menu

• Build LXD or LXC containers:

  • dbmenu || dbmenu --lxc

➡️ Features

• App version upgrades via pypi & Distrobuilder template updates via the Github REST API:

  • dbmenu -v
  • dbmenu -u

• Create:

  • cloud-init per-once / standard configuration
  • template overrides to include custom files / scripts
  • custom templates by merging the template override / cloud-init yaml

• Automatic custom template re-generation as part of updating the standard templates:

  • This ensures custom templates remain in sync with standard templates (which change distribution versions over time)

• Automatic selective caching of json output from LXD images:

  • json read speed improved from 1mb / 0.65 seconds ===> 30kb / 0.0083 seconds
  • Fast yaml reading with yaml.CSafeLoader
  • Fast menu generation (typically 0.03 seconds or less)
  • Auto generated menus for the available container versions your platform can build:

• Version Menu

• Optionally import the built LXD image into incus or lxd
• To disable automatic LXD imports Show User Configuration from the Main Menu & edit / set import_into_lxd to False

➡️ Command line options:

usage: dbmenu [-h] [--lxd | --lxc | -o | -g | -i | -c | -e | -d | -m | -y | -u]
                          [-s] [-t] [--rate] [--reset] [-r] [-v]

Menu driven LXD / LXC images for Distrobuilder

options:
  -h, --help        show this help message and exit
  --lxd             build LXD container / vm image (default)
  --lxc             build LXC container image
  -o, --override    create new template override
  -g, --generate    generate custom template from override
  -i, --init        create / edit cloud-init configuration
  -c, --copy        copy existing template / override
  -e, --edit        edit existing template / override
  -d, --delete      delete template / override
  -m, --move        move / rename template or override
  -y, --merge       merge cloudinit configuration with yq
  -u, --update      force update templates (default auto weekly)
  -s, --show        show configuration settings
  -t, --timer       debug timer used in testing
  --rate            show current Github API Rate Limit
  --reset           reset dbmenu base directory configuration
  -r, --regenerate  regenerate custom templates
  -v, --version     show dbmenu version / update to latest release

➡️ User Configuration:

• User configuration is stored under ~/.config/dbmenu.yaml & is auto generated with sensible defaults on the first run of dbmenu
• The base directory of the distrobuilder area can be optionally changed from the default ~/distrobuilder on first run or at any time via the dbmenu --reset command line option

[~]$ cat ~/.config/dbmenu.yaml
config_dir: /home/stuart/.config
main_dir: /home/stuart/devops/distrobuilder
target_dir: /home/stuart/devops/distrobuilder/build
files_dir: /home/stuart/devops/distrobuilder/files
template_dir: /home/stuart/devops/distrobuilder/templates
cloudinit_dir: /home/stuart/devops/distrobuilder/cloudinit
dbmenu_config: /home/stuart/.config/dbmenu.yaml
gh_owner: lxc
gh_repo: lxc-ci
gh_api_url: https://api.github.com
github_token: ''
cache_dir: false
cleanup: true
compression: xz
console_editor: nano
debug: false
disable_overlay: false
import_into_lxd: true
json_cachefile: /home/stuart/devops/distrobuilder/templates/cache.json
lxd_json: /home/stuart/devops/distrobuilder/templates/lxd.json
lxd_output_type: unified
subdir_custom: /home/stuart/devops/distrobuilder/templates/custom
subdir_images: /home/stuart/devops/distrobuilder/templates/images
subdir_overrides: /home/stuart/devops/distrobuilder/templates/overrides
cloudinit_network_dir: /home/stuart/devops/distrobuilder/cloudinit/network-data
cloudinit_user_dir: /home/stuart/devops/distrobuilder/cloudinit/user-data
cloudinit_vendor_dir: /home/stuart/devops/distrobuilder/cloudinit/vendor-data
timeout: false
yq_check: true

• For normal operation it's not necessary to add a Github Personal Access Token to your User Configuration
• Unauthenticated Github API Rate Limits are not normally exceeded due to connection-pooling in urllib3 & the API calls being made by a singleton instance of Gethub
• To check your current Github API rate limit run dbmenu --rate

➡️ Dependencies

• python 3.10+ / pyyaml / urllib3
• Golang version 4+ of yq (go-yq in Arch Linux)
• incus or lxd
• Distrobuilder version 3.0 or higher

➡️ Installation / Upgrading

• ✅ Arch Linux package from AUR:

  • yay distrobuilder-menu

• ✅ Isolated app (choose either pypi or github):

  • pipx install distrobuilder-menu (pypi)
  • pipx install git+https://github.com/itoffshore/distrobuilder-menu.git (github)
  • size on disk 4mb
  • upgrade release with --force

• ‼️ System module (choose either pypi or github):

  • pip install distrobuilder-menu (pypi)
  • pip install git+https://github.com/itoffshore/distrobuilder-menu.git (github)
  • size on disk 600kb
  • upgrade release with --force

• ⬇️ Upgrading with dbmenu -v

  • 🆕 in version 0.2.5: pip / pipx upgrades
  • 🆕 in version 0.2.8: Arch Linux package detection skips pypi upgrades & displays the latest AUR version

• 🆕 in version 0.2.6 - a compatibility symlink for lxd is automatically created to ensure Distrobuilder can find /var/lib/incus/unix.socket to import built images.

• ⚠️ By default import_into_lxd is True in user settings & can be edited with: dbmenu -s or option 11 from the Main Menu (if neither lxd or incus are installed locally)

  • For versions prior to 0.2.6 create the symlink manually if you use lxd & want to import built images:
  • ln -s /var/lib/lxd /var/lib/incus

❓ Creating Override Templates

dbmenu was inspired by & follows a similar methodology to Hashicorp Packer which builds / creates templates in layers:

• Create a base image override for your chosen distribution with your shell / package customizations that overrides a standard template

• Distribution Menu

• This will generate an override template with an example files & packages sections to customise & optionally open in your configured console_editor (nano by default)

• ⚠️ An override only needs the extra packages you wish to include (& not all of the packages that are included as an example of the yaml from the SOURCE template you are overriding)

• Override Template

• Create a specific override / cloud-init config for your custom service container that contains customizations not in your base image template (e.g web services / database)
• Generate a Custom Template which uses your custom base image template as the SOURCE template & merges your specific overrides / cloud-init for your custom service container

📰 Template Examples

• This repo's examples directory is also packaged under site-packages:

  • e.g for pipx installs:

  • ~/.local/pipx/venvs/distrobuilder-menu/lib/python3.11/site-packages/distrobuilder_menu/examples

  • e.g for installs from an Arch Linux package:

  • /usr/lib/python3.11/site-packages/distrobuilder_menu/examples

• These examples show how to create images for:

  • Alpine Linux / Ubuntu base images
  • Alpine Linux build environment (21mb) that installs alpine-sdk on first boot via cloud-init & most of the steps for contributing packages to Alpine
  • See the alpine-abuild cloud-init bootcmd for how to build an Alpine Linux cloud image & remove cloud-init & it's dependencies on first boot
  • Ubuntu Gitlab container that installs Gitlab on first boot via cloud-init

🏗️ Creating / Building a Custom Template

• Empty input for each menu option / choice will return you to the Main Menu (main event loop)

• Create Custom Override

• Optionally - Create cloud-init Config

• Generate Custom Template

  • this option gives choices to merge a Custom Override & cloud-init configuration
  • you could also just Merge cloud-init Config into an existing template if you only need that option

• Build image - choosing the template type:

  • LXD images are built by default
  • to build lxc images start the app with dbmenu --lxc
  • default container (LXC & LXD)
  • cloud container (LXC & LXD)
  • vm (LXD only)

🏗️ Regenerating Custom Templates (🆕 in v0.2.0)

• Over time the distribution versions in standard Distrobuilder templates change (causing custom templates to become outdated)

• v0.2.0 adds a json footer as a comment with details of how the custom template was generated. To use this new feature existing custom / base templates created before v0.2.0 will need to be re-created (so the json footer is written to the template)

• 🆕 in v0.2.1 - automatic custom template regeneration is incorporated into standard template updates / downloads

• Custom templates can also be regenerated at any time with: dbmenu -r

• Automatic template regeneration

• base templates that use standard templates as a SOURCE are regenerated first
• custom templates which override a base template are regenerated afterwards
• for templates without a dbmenu generated json footer a warning message is shown