devops-agent-adapter 2.3.0

DevOps AI Agent — connects servers to the central orchestration server via Claude Code

DevOps Agent Adapter — EC2 Setup & Testing Guide

This guide explains how to connect a real AWS EC2 instance to your central DevOps Agent server using the pip-installable adapter.

---

Critical Prerequisite: The Networking Problem

The EC2 instance is in AWS. Your central server is on your local PC. AWS cannot reach a private IP on your home/office network. You must first make the central server publicly accessible.

Step 0 — Expose Your Central Server via ngrok

ngrok creates a public HTTPS tunnel to your localhost port.

On your Windows PC (where the backend runs):

# 1. Download ngrok from https://ngrok.com/download (free account)
# 2. Authenticate once (get token from ngrok dashboard)
ngrok config add-authtoken YOUR_NGROK_TOKEN

# 3. Start the tunnel to your Flask backend port
ngrok http 3000

ngrok will output something like:

Forwarding   https://abc123.ngrok-free.app -> http://localhost:3000

Write down this URL — this becomes your central_server value in the identity file. Keep ngrok running for the entire test. Every time you restart ngrok, the URL changes (unless you have a paid plan with a static domain).

Also ensure the Anthropic API key is set on the central server — add it to backend-python/.env before starting the server:

ANTHROPIC_API_KEY=sk-ant-api03-...

The server pushes this key to every node automatically on registration, so you never need to put it in identity.json.

Alternative if you already have a static public IP or domain: just make sure port 3000 is open in your router/firewall and use http://YOUR_PUBLIC_IP:3000.

---

Step 1 — Push Adapter Code to GitHub

The EC2 instance will pip install from your GitHub repo. The adapter is a proper Python package at adapter/ with pyproject.toml.

On your Windows PC:

# From the devops-agent project root
git add adapter/
git commit -m "add pip-installable adapter package"
git push origin main

If your repo is at https://github.com/YOUR_USERNAME/devops-agent, the install command for EC2 will be:

pip install "git+https://github.com/YOUR_USERNAME/devops-agent.git#subdirectory=adapter"

If the repo is private: you'll need a GitHub Personal Access Token (PAT). Install command becomes: pip install "git+https://[e-mail]/YOUR_USERNAME/devops-agent.git#subdirectory=adapter"

---

Step 2 — Connect to EC2 via SSM Session Manager

Open the AWS Console → EC2 → Select an instance → Click Connect → Session Manager tab → Connect.

You get a browser-based terminal. You are initially sh-5.2$ as root.

# Switch to the app user
sudo su - dava-dev-user

---

Step 3 — Prepare the EC2 Instance

The adapter needs Python 3.11+, pip, Node.js, and npm. Check what's installed:

python3 --version      # need 3.11+
pip3 --version
node --version         # need any recent version for Claude Code
npm --version

If Python 3.11+ is missing:

sudo dnf install python3.11 python3.11-pip -y   # Amazon Linux 2023
# or
sudo apt install python3.11 python3.11-pip -y   # Ubuntu

If Node.js/npm is missing:

curl -fsSL https://rpm.nodesource.com/setup_20.x | sudo bash -
sudo dnf install nodejs -y    # Amazon Linux
# or
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo bash -
sudo apt install nodejs -y    # Ubuntu

---

Step 4 — Install the Adapter Package

pip3 install "git+https://github.com/YOUR_USERNAME/devops-agent.git#subdirectory=adapter"

# Verify it installed:
devops-agent --help

You should see:

usage: devops-agent [-h] {init,start,stop,status,logs,install-service,uninstall-service} ...

---

Step 5 — Create the Identity File

# Generate a template
devops-agent init --output ~/identity.json

Now edit it:

nano ~/identity.json

Fill in these values:

{
  "secret": "devops-agent-secret-change-in-prod",

  "node_id": "i-01e3e27652612f067",
  "node_name": "Dev-Server",
  "project_id": "purvansh",
  "project_name": "Purvansh",
  "hostname": "ip-172-31-39-209",
  "environment": "production",

  "central_server": "https://abc123.ngrok-free.app",

  "sandbox": "/home/dava-dev-user",

  "cloud_metadata": {
    "provider": "aws",
    "aws": {
      "credentials": {
        "access_key_id":     "YOUR_ACCESS_KEY_ID",
        "secret_access_key": "YOUR_SECRET_KEY"
      },
      "region":      "us-east-1",
      "account_id":  "",
      "instance_id": "i-01e3e27652612f067"
    }
  },

  "metadata": {
    "server_type": "ec2",
    "version":     "1.0.0",
    "capabilities": [
      "check_service_status",
      "view_logs",
      "check_health",
      "run_command",
      "upload_s3",
      "analyze_code"
    ]
  }
}

Note: anthropic_api_key is no longer set in the identity file. The central server stores it in its .env as ANTHROPIC_API_KEY and automatically pushes it to each node over WebSocket immediately after registration. To rotate the key across all running nodes at once, use the admin API (see Key Rotation below).

Fields to fill in for each instance:

Field	Where to find it
node_id	EC2 Instance ID from AWS Console (e.g. i-01e3e27652612f067)
node_name	The Name tag of the EC2 instance (e.g. Dev-Server)
central_server	The ngrok URL from Step 0
cloud_metadata.aws.credentials	The IAM agent role keys stored in your project
sandbox	The home directory of the user, e.g. /home/dava-dev-user

secret must match NODE_SECRET in your central server's config.py.

---

Step 6 — Start the Adapter

The adapter runs as a background daemon by default — it won't block your terminal.

devops-agent start --identity ~/identity.json

Expected output:

🚀 Starting DevOps Agent adapter (daemon)
   Node    : web-server  (i-071956daacfd55a0f)
   Project : Purvansh  (purvansh)
   Server  : https://abc123.ngrok-free.app
   Sandbox : /root

✅ Daemon started (PID 12345)
   Logs : /var/log/devops-agent/adapter.log

   Commands:
     devops-agent status   — check if running
     devops-agent logs     — view live logs
     devops-agent stop     — stop the daemon

Your terminal is immediately free for other work. The adapter runs in the background.

Foreground mode (for debugging)

If you need to see live output in the terminal (useful for debugging):

devops-agent start --identity ~/identity.json --foreground

Press Ctrl+C to stop when running in foreground mode.

---

Step 7 — Managing the Adapter

Check Status

devops-agent status

Output:

🟢 DevOps Agent is running (PID 12345)
   Logs: /var/log/devops-agent/adapter.log

or:

🔴 DevOps Agent is not running.

View Logs

# Show last 50 lines
devops-agent logs

# Follow logs in real-time (like tail -f)
devops-agent logs --follow

# Show last 100 lines
devops-agent logs --lines 100

Stop the Adapter

devops-agent stop

Output:

🛑 Stopping DevOps Agent (PID 12345)...
✅ Stopped.

The node will show as offline on the dashboard.

---

Step 8 — Auto-Start on Boot (Systemd Service)

To ensure the adapter automatically starts whenever the EC2 instance boots or reboots:

sudo devops-agent install-service --identity ~/identity.json

This will:

1. Create a systemd service file at /etc/systemd/system/devops-agent.service
2. Enable it to start on boot
3. Optionally start it immediately

After installing:

# Check service status
sudo systemctl status devops-agent

# Manually start/stop/restart
sudo systemctl start devops-agent
sudo systemctl stop devops-agent
sudo systemctl restart devops-agent

# View service logs
devops-agent logs --follow
# or
journalctl -u devops-agent -f

Remove the service

sudo devops-agent uninstall-service

This stops the service, disables it, and removes the unit file.

Important: When running as a systemd service, if the EC2 instance is stopped, the adapter stops too and the node shows offline. When the instance starts again, the service auto-starts and the node comes back online automatically.

---

Step 9 — Verify on Dashboard

1. Open your browser → Dashboard
2. You should see the project card with 1/1 nodes online
3. Sidebar NODES section should show the node with a green dot

---

Step 10 — Run a Test Command

Click on the project card → opens the AI console.

Type:

check disk usage on Dev-Server

Expected flow:

• Job card appears (running)
• Central server logs: 🚀 Job dispatched to i-01e3e27652612f067
• Claude Code runs df -h on the EC2 instance
• Result appears in the job card with actual disk usage

---

Key Rotation

The Anthropic API key lives only on the central server (backend-python/.env → ANTHROPIC_API_KEY). To rotate it across all live nodes without touching any EC2 instance:

Option A — Update .env and restart the server (new nodes get the new key on next registration).

Option B — Hot-rotate without any restarts (all currently connected nodes are updated instantly):

# Get an admin JWT first (login via the dashboard or API)
TOKEN="<your-admin-jwt>"

curl -X POST https://abc123.ngrok-free.app/api/admin/config/update \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"anthropic_api_key": "sk-ant-NEW-KEY-HERE"}'

Expected response:

{"ok": true, "nodes_updated": 3}

Each connected node immediately logs:

🔑 [config_update] ANTHROPIC_API_KEY updated from central server

The next claude job run on every node uses the new key. No SSH, no SSM, no restarts required.

---

Repeating for More Instances

For each additional EC2 instance:

1. SSM into that instance
2. pip3 install ... (same command)
3. devops-agent init --output ~/identity.json
4. Edit identity.json — change only node_id and node_name (no anthropic_api_key needed)
5. devops-agent start --identity ~/identity.json
6. (Optional) sudo devops-agent install-service --identity ~/identity.json for auto-start

---

Log File Locations

User	Log File	PID File
root	/var/log/devops-agent/adapter.log	/var/run/devops-agent.pid
non-root	~/.devops-agent/logs/adapter.log	~/.devops-agent/devops-agent.pid

Logs rotate automatically at 10 MB with 5 backups kept, so they won't fill your disk.

---

CLI Quick Reference

Command	Description
devops-agent init -o ~/identity.json	Generate template identity file
devops-agent start -i ~/identity.json	Start adapter as background daemon
devops-agent start -i ~/identity.json --foreground	Start in foreground (debug mode)
devops-agent stop	Stop the running daemon
devops-agent status	Check if adapter is running
devops-agent logs	Show last 50 log lines
devops-agent logs -f	Tail logs in real-time
devops-agent logs -n 200	Show last 200 log lines
sudo devops-agent install-service -i ~/identity.json	Install systemd service
sudo devops-agent uninstall-service	Remove systemd service

---

Troubleshooting

Symptom	Cause	Fix
Connection refused	ngrok not running or URL changed	Restart ngrok, update central_server in identity.json
AUTH_FAILED in logs	Wrong secret	Check NODE_SECRET in config.py matches "secret" in identity.json
Node doesn't appear in dashboard	Wrong project_id	Must match exactly (case-sensitive)
devops-agent: command not found	pip installed to non-PATH location	Use python3 -m devops_agent.cli start ... instead
Claude Code install fails	npm missing	Install Node.js first (Step 3)
No [config_update] log after registration	ANTHROPIC_API_KEY not set on the server	Add ANTHROPIC_API_KEY=sk-ant-... to backend-python/.env and restart the server
Claude returns auth error during job	Key not received or empty	Check server logs for 🔑 API key pushed to <node_id>; verify .env has the key
⚠️ DevOps Agent is already running	Previous instance still running	Run devops-agent stop first
Permission denied on stop/install	Need root	Prefix with sudo
Node offline after EC2 reboot	Service not installed	Run sudo devops-agent install-service -i ~/identity.json

---

Files Reference

• devops_agent/cli.py — CLI entry point (init, start, stop, status, logs, install-service, uninstall-service)
• devops_agent/runner.py — socket agent logic (uses Python logging)
• identity.example.json — full template reference