Personal Automation Toolkit (auto-toolkit)

A modular, discoverable, and extensible automation engine and CLI for general-purpose scripts, system administration utilities, and application lifecycle management.

The toolkit provides a single unified $PATH command—auto—that acts as the entry point for listing, searching, discovering, and executing automations across single or multiple repositories from any working directory.

Table of Contents

1. Features
2. Multi-Repository & Extensibility Model
3. Security Notice & Repository Trust Model
4. Installation
5. CLI Usage
   • Discover Automations
   • Search Automations
   • Multi-Repo Commands
   • Inspect Automation Details
   • Execute Automations
   • Dedicated Application Management
   • Repository Health Validation
6. Repository Structure
7. Shared Libraries (data/libs/)
8. Architecture & How It Works
9. Creating a New Automation or App
10. Development & Testing

Features

• Single Global CLI (auto): Access all automations seamlessly from any working directory.
• Multi-Repository Discovery: Extend auto with personal, team, or private repositories without modifying or dirtying the core engine repository.
• Repository Trust Verification: First-class security model that prompts and remembers trusted repositories by remote URL and branch.
• Convention Over Configuration: No central registry files. Adding an executable file under data/scripts/<category>/<name> or data/apps/<app>/<action> automatically makes it available.
• Dedicated Application Management: Clean verb-noun commands (auto install <app>, auto update <app>) for apps requiring complex multi-step workflows like RPM/DEB downloads and cryptographic PGP signature verification.
• Shared Helpers Library (data/libs/): Declarative CLI argument parsing (args.sh), isolated GPG signature verification, robust downloading, and package management.
• Language Agnostic: Automations can be written in Bash, Python, Go, Rust, Ruby, or compiled binaries.
• Transparent Execution: Arguments, stdin/stdout/stderr, exit status, and signals are forwarded without subshell wrapping via os.execvp.
• Zero Production Dependencies: Pure standard-library Python core.
• Rich Discovery & Ranked Search: Fast multi-token search scoring across script names, descriptions, tags, dependencies, and authors.
• Built-in Validation: auto validate diagnoses file permissions, shebang errors, missing dependencies, and duplicate names.
• Lifecycle & Self-Updating: Built-in auto self-update and auto repo-update to maintain the core engine and external repositories.

Multi-Repository & Extensibility Model

auto-toolkit separates the core execution engine from user automation scripts:

┌────────────────────────────────────────────────────────┐
│                   auto CLI Engine                      │
└──────────────────────────┬─────────────────────────────┘
                           │
             ┌─────────────┴─────────────┐
             ▼                           ▼
┌─────────────────────────┐ ┌─────────────────────────┐
│       Core Repo         │ │   Personal / Team Repo  │
│     (auto-toolkit)      │ │   (e.g. ~/.auto-toolkit/repos/my-autos) │
├─────────────────────────┤ ├─────────────────────────┤
│ • src/ CLI Engine       │ │ • Personal scripts      │
│ • Starter shell libs    │ │ • Company utilities     │
│ • Base automations      │ │ • Private app scripts   │
└─────────────────────────┘ └─────────────────────────┘

Register user repositories via ~/.auto-toolkit/config.toml or AUTO_REPOS env var:

# ~/.auto-toolkit/config.toml
[repos]
personal = "~/.auto-toolkit/repos/personal-automations"
work = "~/.auto-toolkit/repos/work-automations"

Security Notice & Repository Trust Model

Safety & Trust Guarantees

1. Core Repository (core):

   • The core toolkit repository (auto-toolkit) is trusted unconditionally.

2. External / Personal Repositories:

   • When executing scripts (auto run <name>), installing apps (auto install <app>), or updating (auto repo-update <name>) from an external repository for the first time, auto displays a security notice detailing the repository name, path, Git branch, and remote URL.
   • You will be prompted to confirm trust ([y/N]).
   • Upon confirmation, the repository fingerprint (repo_path, remote_url, branch) is saved to ~/.auto-toolkit/trusted_repos.json.
   • Never Warn Again: Once confirmed as trusted, auto remembers it permanently and will not prompt again for that repository.

3. Tamper & Remote Divergence Protection:

   • If a repository's Git remote URL or branch changes (e.g. switched to a new branch or remote), auto detects the change, invalidates prior trust, and requires a new confirmation.
   • All repository pulls use git pull --ff-only to guarantee that local branches cannot have unreviewed merge commits automatically injected.

Registering User Repositories

You can connect external repositories using either ~/.auto-toolkit/config.toml or the AUTO_REPOS environment variable.

Option A: Configuration File (~/.auto-toolkit/config.toml)

[repos]
personal = "~/.auto-toolkit/repos/automations"
work = "~/work/team-automations"

Option B: Environment Variable (AUTO_REPOS)

export AUTO_REPOS="personal=~/.auto-toolkit/repos/automations:work=~/work/team-automations"

When multiple repositories are active:

• auto repos lists all registered sources.
• auto list and auto search display source tags (e.g. [core], [personal]).
• auto repo-update updates all non-core user repositories with a single command.

Installation

1. Direct Symlink / Local Installation (Recommended)

Clone the repository and run the application installer or self-install CLI:

git clone https://github.com/othonhugo/auto-toolkit.git
cd auto-toolkit
./data/apps/auto-toolkit/install.sh

Or run via the CLI itself:

./bin/auto self-install

2. Custom Installation Directory

./data/apps/auto-toolkit/install.sh --path /usr/local/bin
# or
./bin/auto self-install --path /usr/local/bin

3. Ensure $PATH is Configured

export PATH="$HOME/.local/bin:$PATH"

Verify the installation:

auto version

4. Uninstallation

auto uninstall

CLI Usage

1. Manage Registered Repositories

# List all active automation repositories
auto repos

# Update core engine repository
auto self-update

# Update all external user repositories (or a specific one)
auto repo-update
auto repo-update personal

2. List Available Automations

# List all automations grouped by category
auto list

# List automations in a specific category
auto list docker
auto list setup
auto list apps/auto-toolkit

# List all categories and counts
auto categories

3. Search Automations

Search by keyword across names, descriptions, tags, and dependencies:

auto search docker
auto search cleanup
auto search git

4. Inspect Automation Metadata

Inspect documentation, requirements, paths, and usage without running the script:

auto info setup-dev-env
auto info auto-toolkit/install

5. Execute Automations

Execute automations using shorthand or the explicit run command:

# Shorthand execution (transparent argument forwarding)
auto setup-dev-env --dry-run
auto find-large-files /var/log --min-size 50M --top 10
auto hash-check /path/to/file --algo sha256

# Explicit run command
auto run docker-cleanup --dry-run

6. Dedicated Application Management (data/apps/)

# List all dedicated applications and their available actions
auto apps

# Install a specific application
auto install auto-toolkit
auto install auto-toolkit --dry-run
auto install auto-toolkit --path ~/.local/bin

# Check and update a specific application
auto update auto-toolkit
auto update auto-toolkit --check-only

7. Repository Health Validation

Run diagnostics across all registered automation repositories:

auto validate

Repository Structure

auto-toolkit/
├── bin/
│   └── auto                  # Executable CLI entrypoint
├── src/                      # CLI implementation
│   ├── __init__.py
│   ├── cli.py                # Command routing and terminal presentation
│   ├── colors.py             # ANSI terminal color utilities
│   ├── config.py             # Multi-repo configuration loader
│   ├── constants.py          # Central paths, extensions, and exclusions
│   ├── discovery.py          # Dynamic file scanner and alias resolver
│   ├── enums.py              # Strongly-typed HeaderKey & Severity enums
│   ├── executor.py           # Pre-flight checks and os.execvp replacement
│   ├── installer.py          # Symlink installer and PATH validator
│   ├── metadata.py           # Header comment parser (DESCRIPTION, TAGS, etc.)
│   ├── search.py             # Multi-token ranked search engine
│   └── validator.py          # Repository health and diagnostic checks
├── data/                     # Executable assets & libraries
│   ├── apps/                 # Dedicated application packages
│   ├── scripts/              # Category-organized general utilities
│   └── libs/                 # Reusable shell helpers
│       ├── args.sh           # Declarative CLI argument parsing library
│       ├── common.sh         # Logging, formatting, and temp dirs
│       ├── download.sh       # Robust curl/wget downloads
│       ├── gpg.sh            # GPG key fetch & detached signature verification
│       └── pkg.sh            # RPM/DEB package manager abstractions
├── tests/                    # Unit and integration test suite
├── Makefile                  # Developer shortcuts
├── pyproject.toml            # Build metadata & PyPI package definition
└── README.md                 # Documentation

Shared Libraries (data/libs/)

Reusable helper libraries available for all automations and user repositories:

Architecture & How It Works

CLI Invocation  ──>  Multi-Repo Config  ──>  Discovery  ──>  Metadata  ──>  Resolution / Search  ──>  os.execvp Execution
(auto <target>)      (Env / config.toml)     (Multi-root)    (Headers)     (Category/Name/Alias)      (Process handover)

1. Multi-Root Dynamic Discovery

• Inspects data/scripts/ and data/apps/ across all registered repositories.
• Strips script extensions (.sh, .py, .go, etc.) for clean CLI invocation.
• Creates ergonomic aliases for apps (install-<app>, <app>/install, <app>/update).

2. Process Execution (os.execvp)

• Runs pre-flight verification (executable permissions and declared $PATH dependencies).
• Transparently replaces the Python CLI process with the target executable.
• Preserves full TTY interactivity, stdout/stderr streams, signal handling (SIGINT), and exact child exit codes.

3. Metadata Conventions

Automations declare metadata using lightweight header comments near the top of the file:

Creating a New Automation or App

Adding a General Automation

Create an executable script under data/scripts/<category>/<name>.sh:

#!/usr/bin/env bash
# DESCRIPTION: Check SSL certificate expiration date
# TAGS: network, ssl, tls, security
# DEPENDS: openssl
# USAGE: auto check-ssl <domain>

set -euo pipefail
DOMAIN="${1:?Please provide a domain}"
openssl s_client -servername "$DOMAIN" -connect "$DOMAIN:443" </dev/null 2>/dev/null | openssl x509 -noout -dates

Make it executable:

chmod +x data/scripts/network/check-ssl.sh

Adding a Dedicated App

Create data/apps/<app-name>/install.sh and data/apps/<app-name>/update.sh:

#!/usr/bin/env bash
# DESCRIPTION: Install MyTool with PGP verification
# TAGS: app, mytool, install, binary
# DEPENDS: curl, gpg

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
LIB_DIR="$(cd "$SCRIPT_DIR/../../libs" && pwd)"
source "$LIB_DIR/common.sh"
source "$LIB_DIR/download.sh"
source "$LIB_DIR/gpg.sh"

Make it executable:

chmod +x data/apps/<app-name>/*.sh

Development & Testing

# Run full test suite
uv run pytest -v

# Run linting and code formatting checks
uv run ruff check src tests

# Build distribution packages
uv build