raspi-sentinel 0.9.0

Standalone staged self-healing layer for Raspberry Pi systemd services

raspi-sentinel

raspi-sentinel is a guarded recovery supervisor for Raspberry Pi services managed by systemd.

It detects logical stalls (stale files, failed command checks, stale status JSON, dependency failures) and applies staged recovery:

warn -> restart services -> guarded reboot

Japanese guide: README.ja.md

Open Beta: v0.9.x

v0.9.x is the open beta line before v1.0.0. Current stable released line is v0.8.x. Release target for this cycle is v0.9.0.

Who should try this:

• You run Raspberry Pi services with systemd
• You can inspect logs and edit TOML
• You are willing to start with dry-run mode
• You can report reproducible failures through GitHub Issues

Do not use yet if:

• You need unattended production recovery from day one
• Unexpected reboot would be dangerous
• You cannot access the machine physically or via fallback path
• You need fleet-level centralized monitoring

Which Version To Use

Stable release: v0.8.0

Use this if you want the latest released version:

git clone https://github.com/yukimurata0421/raspi-sentinel.git
cd raspi-sentinel
git checkout v0.8.0

For v0.8.0 usage details, follow the README included in that tag.

Open beta line (v0.9.x)

Before v0.9.0 tag is published, use:

git clone https://github.com/yukimurata0421/raspi-sentinel.git
cd raspi-sentinel
git checkout main

After v0.9.0 is tagged, always use git checkout v0.9.0 for reproducibility.

What This Does

• Monitors local service health while OS is alive
• Evaluates file freshness, command checks, service active state, semantic stats, external status JSON, dependency probes, optional clock checks
• Applies deterministic staged recovery with cooldown/window guards
• Persists inspectable state/events for postmortem

Network-only failures (DNS/gateway path) are excluded from direct reboot reasons by default.

Output Model

raspi-sentinel separates runtime outputs by role:

• state.json: recovery-oriented engine state
• stats.json: current operational snapshot for operators and integrations
• events.jsonl: append-only transition/audit trail for postmortem

What This Does Not Do

• Hardware watchdog replacement
• Full kernel/hardware hang detector
• Fleet monitoring control plane
• Internet-facing/multi-tenant control plane
• Untrusted command execution tool

Safety Model

raspi-sentinel assumes trusted operator-controlled config on a single machine.

Recommended config ownership:

sudo chown root:root /etc/raspi-sentinel/config.toml
sudo chmod 0600 /etc/raspi-sentinel/config.toml

Why this matters:

• Config may contain webhook URLs and command arguments
• Recovery actions may restart services or request reboot
• shell=False is default; shell execution is explicit opt-in

15-Minute Beta Demo (Non-root Friendly)

1. Clone beta line

git clone https://github.com/yukimurata0421/raspi-sentinel.git
cd raspi-sentinel
git checkout main

This beta demo relies on v0.9.x line files. Before the v0.9.0 tag is published, use main.

2. Install

python3 -m pip install .

Optional (pipx) install path for CLI trial:

pipx install "git+https://github.com/yukimurata0421/raspi-sentinel.git@main"

3. Prepare demo workspace and config (no restart/reboot, no notifications)

install -d -m 0755 /tmp/raspi-sentinel-demo
cp config/raspi-sentinel.beta-demo.toml /tmp/raspi-sentinel-demo/config.toml
${EDITOR:-vi} /tmp/raspi-sentinel-demo/config.toml

4. Validate config

raspi-sentinel -c /tmp/raspi-sentinel-demo/config.toml validate-config --strict

5. Run doctor

raspi-sentinel -c /tmp/raspi-sentinel-demo/config.toml doctor --json

6. Initialize demo heartbeat (healthy baseline)

python3 scripts/failure_inject.py fresh-file --path /tmp/raspi-sentinel-demo/heartbeat.txt

7. Run one dry-run cycle (expected healthy)

raspi-sentinel -c /tmp/raspi-sentinel-demo/config.toml --dry-run run-once --json

8. Inject sample failure

python3 scripts/failure_inject.py stale-file --path /tmp/raspi-sentinel-demo/heartbeat.txt --age-sec 900

Then run dry-run again:

raspi-sentinel -c /tmp/raspi-sentinel-demo/config.toml --dry-run run-once --json

9. Inspect and stop

tail -n 20 /tmp/raspi-sentinel-demo/events.jsonl
raspi-sentinel -c /tmp/raspi-sentinel-demo/config.toml explain-state --json
sudo systemctl disable --now raspi-sentinel.timer

10. Clean up demo files (optional)

rm -rf /tmp/raspi-sentinel-demo

Production Setup (Root-owned Config)

For production/systemd operation, install config under /etc:

sudo install -d -m 0755 /etc/raspi-sentinel
sudo install -m 0600 -o root -g root config/raspi-sentinel.example.toml /etc/raspi-sentinel/config.toml
sudo "${EDITOR:-vi}" /etc/raspi-sentinel/config.toml
raspi-sentinel -c /etc/raspi-sentinel/config.toml validate-config --strict

Emergency Stop

sudo systemctl disable --now raspi-sentinel.timer
sudo systemctl stop raspi-sentinel.service

Enable Timer (After Dry-run Verification)

Install units using helper (renders ExecStart with detected raspi-sentinel binary path):

BIN="$(command -v raspi-sentinel)"
sudo python3 scripts/install_systemd.py --raspi-sentinel-bin "$BIN" --enable-timer

Recommended systemd-visible install path (/opt) example:

sudo install -d -m 0755 /opt/raspi-sentinel
sudo chown "$USER":"$USER" /opt/raspi-sentinel
python3 -m venv /opt/raspi-sentinel/.venv
/opt/raspi-sentinel/.venv/bin/pip install .
sudo python3 scripts/install_systemd.py \
  --raspi-sentinel-bin /opt/raspi-sentinel/.venv/bin/raspi-sentinel \
  --enable-timer

The helper is recommended because it renders ExecStart to your actual binary path. The bundled service uses ProtectHome=true, so avoid /home/... binary paths. Prefer /opt/raspi-sentinel/.venv/bin/raspi-sentinel or another system-visible path. Manual installation is possible, but it is not equivalent unless you install all required units and edit ExecStart yourself:

sudo install -m 0644 systemd/raspi-sentinel.service /etc/systemd/system/raspi-sentinel.service
sudo install -m 0644 systemd/raspi-sentinel.timer /etc/systemd/system/raspi-sentinel.timer
sudo install -m 0644 systemd/raspi-sentinel-tmpfs-verify.service /etc/systemd/system/raspi-sentinel-tmpfs-verify.service
sudo systemctl daemon-reload
sudo systemctl enable --now raspi-sentinel.timer

If [storage].require_tmpfs = true or tmpfs tiering is configured, include mount unit install:

BIN="$(command -v raspi-sentinel)"
sudo python3 scripts/install_systemd.py --raspi-sentinel-bin "$BIN" --include-tmpfs-mount --enable-timer

--dry-run disables restart/reboot and suppresses external notifications by default. Use --send-notifications only when you intentionally want notification path testing in dry-run.

When running under the bundled systemd service, ProtectHome=true is enabled. Paths under /home can work in manual CLI dry-run but fail when timer execution starts.

Feedback Wanted (v0.9.x Open Beta)

Please report:

• install failures
• confusing config validation / doctor output
• dry-run behavior that looks unsafe or unclear
• false positives / false negatives
• systemd timer/service integration issues
• failure injection outcomes
• docs gaps

Use issue form:

• Open beta feedback issue

Helpful report commands:

raspi-sentinel --version
python3 --version
systemctl --version
raspi-sentinel -c /etc/raspi-sentinel/config.toml validate-config --strict
raspi-sentinel -c /etc/raspi-sentinel/config.toml doctor --json
raspi-sentinel -c /etc/raspi-sentinel/config.toml --dry-run run-once --json
raspi-sentinel -c /etc/raspi-sentinel/config.toml doctor --json --support-bundle ./support-bundle.json

Do not post secrets in public issues:

• webhook URLs
• tokens
• private hostnames
• sensitive local paths

Documentation Map

Start here:

• Documentation Index
• Upgrade Guide
• Security Policy
• Output Contract
• Storage Tiers
• Watchdog Integration
• Time Health Decision Table
• Operations Runbook

Tests and CI details: docs/facts/test-map.md

Versioning

Current stable line: v0.8.x. Open beta line: v0.9.x.

See:

• CHANGELOG.md
• docs/VERSIONING.md

License

MIT License.