mocea 2.0.4

Idle droplet monitor and auto-terminator.

mocea

Idle droplet monitor and auto-terminator.

mocea (Monitoring and Management of digital OCEA droplets) is a lightweight agent that runs on a DigitalOcean droplet, detects when it's idle, and auto-terminates it via the DigitalOcean API (snapshot + destroy) to save costs. It runs as a systemd service and uses psutil for rich local metric collection.

Installation

pip install mocea

Quick Start

# Test the checks
mocea check

# Start in foreground with dry-run
mocea run --dry-run --idle-minutes 5

# Install as systemd service
sudo mocea install

CLI Commands

mocea run          Start the monitoring agent
mocea check        Run all checks once and display results
mocea status       Show service status and current check results
mocea config       Show active configuration
mocea cloudinit    Generate cloud-init user-data for droplet bootstrap
mocea install      Install mocea as a systemd service
mocea uninstall    Remove mocea systemd service

mocea run

Options:
  -c, --config PATH       Config file path
  --idle-minutes INTEGER  Override idle timeout (minutes)
  --dry-run               Log actions but don't execute
  --log-level [DEBUG|INFO|WARNING]

Configuration

Config file: /etc/mocea/config.toml (or ~/.config/mocea/config.toml)

idle_minutes = 30
check_interval = 60
min_uptime_minutes = 10

[checks.cpu]
enabled = true
threshold = 5.0          # percent, below = idle

[checks.process]
enabled = true
names = ["python", "ffmpeg", "jupyter"]

[checks.ssh]
enabled = true

[checks.load]
enabled = false
threshold = 0.3

[checks.network]
enabled = false
threshold_kbps = 10
# interface = "eth0"      # omit for all non-loopback interfaces

[checks.gpu]
enabled = false

[checks.heartbeat]
enabled = false
file = "/tmp/mocea-heartbeat"
stale_minutes = 15

[action]
type = "api"              # "api" | "shutdown"

# Optional: api action settings
# [action.api]
# snapshot_before_destroy = true

[logging]
level = "INFO"
# file = "/var/log/mocea.log"  # omit for stdout only

Priority: CLI flags > config file > defaults.

Checks

Check	Signal	Idle when
cpu	psutil.cpu_percent()	Below threshold (default 5%)
process	psutil.process_iter()	No configured process names running
ssh	psutil.net_connections() port 22	No ESTABLISHED SSH sessions
load	psutil.getloadavg()	1-min load below threshold (default 0.3)
network	psutil.net_io_counters()	Throughput below threshold (default 10 KB/s)
gpu	nvidia-smi	GPU utilization below threshold (default 5%)
heartbeat	File mtime	File missing or stale (default 15 min)

All enabled checks must report idle (AND logic) for the entire idle_minutes duration before the action triggers.

Actions

Action	Description
api	Direct DO API: snapshot then destroy (default, requires DO_API_TOKEN)
shutdown	Simple shutdown -h now (no snapshot, for disposable droplets)

Safety

• Minimum uptime: Won't terminate within first 10 minutes after boot
• Dry-run mode: --dry-run logs what would happen without executing
• Lock file: Prevents multiple instances
• Fail-safe: If a check errors, it's treated as "active" (keeps running)
• Heartbeat file: Any process can touch /tmp/mocea-heartbeat to prevent termination

Managing the Systemd Service

Once installed with sudo mocea install, mocea runs as a standard systemd service called mocea.service. Here's how to manage it.

Checking Status

# mocea's built-in status (service + check results)
sudo mocea status

# Or use systemctl directly
sudo systemctl status mocea.service

Viewing Logs

mocea logs to the systemd journal. Use journalctl to read them:

# Recent logs
sudo journalctl -u mocea.service -n 50

# Follow logs in real time (like tail -f)
sudo journalctl -u mocea.service -f

# Logs since last boot
sudo journalctl -u mocea.service -b

# Logs from the last hour
sudo journalctl -u mocea.service --since "1 hour ago"

Stopping and Starting

# Stop mocea (it will restart on next boot since it's still enabled)
sudo systemctl stop mocea.service

# Start it again
sudo systemctl start mocea.service

# Restart (stop + start)
sudo systemctl restart mocea.service

Disabling and Re-enabling

"Enabled" means mocea starts automatically on boot. "Disabled" means it won't.

# Stop AND prevent auto-start on boot
sudo systemctl disable --now mocea.service

# Re-enable auto-start (and start it now)
sudo systemctl enable --now mocea.service

Updating mocea

mocea is installed in a virtualenv at /opt/mocea. To upgrade, use the venv's pip — not the system pip (which will fail with externally-managed-environment on modern Debian/Ubuntu):

# Stop the service
sudo systemctl stop mocea.service

# Upgrade using the venv's pip
sudo /opt/mocea/bin/pip install --upgrade --no-cache-dir mocea

# Restart the service to pick up the new code
sudo systemctl start mocea.service

# Verify the new version
mocea --version

If the mocea binary path changed (e.g. you recreated the virtualenv), reinstall the service:

sudo mocea uninstall
sudo /path/to/new/venv/bin/mocea install

Uninstalling

sudo mocea uninstall

This stops the service, disables it, and removes the unit file.

Troubleshooting

Symptom	Command	What to look for
Service won't start	sudo journalctl -u mocea.service -n 30	Python errors, missing config
Exit code 203/EXEC	cat /etc/systemd/system/mocea.service	ExecStart must be an absolute path; reinstall with sudo mocea install
Service keeps restarting	sudo systemctl status mocea.service	Shows restart count and exit code
Not sure if running	sudo systemctl is-active mocea.service	Prints active or inactive

The service is configured to restart automatically on failure after a 30-second delay. If it keeps crashing, check the logs with journalctl.

Cloud-Init Bootstrap

Generate cloud-init user-data to auto-install mocea on new droplets:

mocea cloudinit > user-data.yaml

Development

git clone https://github.com/ksteptoe/mocea.git
cd mocea
make bootstrap    # Create venv and install dependencies
make test         # Run tests
make lint         # Run linter
make format       # Format code