Skip to main content

GCE Rescue - Rescue unbootable Google Compute Engine VMs.

Project description

GCE Rescue

test badge

Rescue unbootable Google Compute Engine VMs by swapping disks on the same VM — no new instance created, same IP, no data loss. Creates a safety snapshot before any changes.

Auto-fix path: The repair command reads serial console output, identifies the boot failure, and applies a fix automatically end to end.

Rescue path: When auto-fix is not available for the detected issue, the rescue command swaps your broken boot disk with a rescue disk and attaches the original boot disk as a secondary disk, providing a rescue environment for manual repair. Once fixed, the restore command puts your fixed boot disk back.

gce-rescue diagnose my-vm --zone=us-central1-a    # What's wrong?
gce-rescue repair my-vm --zone=us-central1-a      # Auto-fix it

GCE Rescue Workflow

Note: GCE Rescue is not an officially supported Google Cloud product. The Google Cloud Support team maintains this repository.

Requirements: Python >= 3.9, gcloud CLI, roles/compute.instanceAdmin.v1 IAM role.

Installation

Google Cloud Shell (recommended)

Open Cloud Shell — Python, gcloud, and authentication are already set up.

pip install git+https://github.com/GoogleCloudPlatform/gce-rescue.git
Local Machine

Linux / macOS

curl -sSL https://raw.githubusercontent.com/GoogleCloudPlatform/gce-rescue/main/install.sh | bash

May require sudo if Python or pip is not installed.

Windows (run PowerShell as Administrator)

irm https://raw.githubusercontent.com/GoogleCloudPlatform/gce-rescue/main/install.ps1 | iex

The installers handle all prerequisites (Python, gcloud, PATH, authentication) and will prompt before installing anything.


Install from source (requires Python >= 3.9, gcloud CLI, Git)

git clone https://github.com/GoogleCloudPlatform/gce-rescue.git
cd gce-rescue
pip install .

Authentication

Environment Setup
Cloud Shell Pre-authenticated, nothing to do
GCE VM (with service account) Automatic via metadata server
GCE VM (without compute scopes) gcloud auth application-default login
Local machine gcloud auth application-default login

More info: Application Default Credentials

Usage

Command What it does Modifies VM?
diagnose Identifies boot errors from serial console output No
repair Diagnoses and fixes boot issues automatically Yes
rescue Provides a rescue environment for investigation via SSH/RDP Yes
restore Reverses rescue, puts your fixed boot disk back Yes
  1. Start with diagnose — understand what's wrong (safe, read-only)

    gce-rescue diagnose VM_NAME --zone=ZONE
    
  2. Auto-fix available? — let repair handle it automatically

    gce-rescue repair VM_NAME --zone=ZONE
    
  3. Need manual access? — enter rescue mode, fix it yourself

    gce-rescue rescue VM_NAME --zone=ZONE
    # SSH/RDP in, fix the issue on /mnt/sysroot
    gce-rescue restore VM_NAME --zone=ZONE
    

All operations create a snapshot before changes, roll back automatically on failure, and can resume if interrupted.

Sample output: diagnose
$ gce-rescue diagnose my-vm --zone=us-central1-a
Diagnosis: my-vm (us-central1-a)
Status:    RUNNING
OS:        Linux (debian-12-bookworm, x86_64, Free)
Result:    Found 1 boot error(s)

  [fstab_bad_uuid] Bad UUID in /etc/fstab (critical)
    Line: UUID=abcd-1234  /data  ext4  defaults  0  2
    Fix:  Remove or correct the fstab entry, then reboot

  Recommended: gce-rescue repair my-vm --zone=us-central1-a

Flags

Flag Description
--zone GCP zone (required)
--project GCP project (default: current gcloud config)
--no-snapshot Skip safety snapshot (faster)
--quiet No confirmation prompts (for automation)
--format Output format: json, yaml, table

Features

Feature Description
Linux + Windows Auto-detects OS, uses appropriate rescue environment
Boot Diagnostics Serial console analysis for fstab, GRUB, kernel, filesystem errors
Auto-Repair Automated fix for fstab errors (more categories planned)
Automatic Rollback Operations roll back on failure
Session Recovery Resume or rollback interrupted operations
Safety Snapshots Backup snapshot before any changes (default)
ARM64 Support Automatic architecture detection

Permissions

roles/compute.instanceAdmin.v1 includes all permissions needed for every command.

Command Minimum Role
diagnose roles/compute.viewer
rescue, restore, repair roles/compute.instanceAdmin.v1
gcloud projects add-iam-policy-binding PROJECT_ID \
    --member="user:EMAIL" \
    --role="roles/compute.instanceAdmin.v1"
Full permissions matrix
Permission diagnose repair rescue restore
compute.projects.get x x x x
compute.instances.get x x x x
compute.instances.getSerialPortOutput x x x
compute.instances.stop x x x
compute.instances.start x x x
compute.instances.attachDisk x x x
compute.instances.detachDisk x x x
compute.instances.setMetadata x x x
compute.disks.create x x
compute.disks.delete x x x
compute.disks.get x x x
compute.disks.createSnapshot x* x*
compute.snapshots.create x* x*
compute.snapshots.get x* x*
compute.snapshots.list x x
compute.snapshots.delete x* x*

* Skippable with --no-snapshot

V1 Legacy

V1 is available as gce-rescue-v1 for backward compatibility:

gce-rescue-v1 -n VM_NAME -z ZONE -p PROJECT

See the V1 documentation for details.

Uninstall

pip uninstall gce-rescue

# Linux/macOS (if installed via install script)
rm -rf ~/.gce-rescue

Contact

GCE Rescue Team: gce-rescue-dev@google.com

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gce_rescue-2.0.0.tar.gz (179.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

gce_rescue-2.0.0-py3-none-any.whl (238.6 kB view details)

Uploaded Python 3

File details

Details for the file gce_rescue-2.0.0.tar.gz.

File metadata

  • Download URL: gce_rescue-2.0.0.tar.gz
  • Upload date:
  • Size: 179.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.7

File hashes

Hashes for gce_rescue-2.0.0.tar.gz
Algorithm Hash digest
SHA256 4d3c65b9fc369bb3890e2a4ee668b8b6e50bd300bd07a9649aaad7e05b2362fc
MD5 ed09034bf108c51c193cc3cb9bcb9782
BLAKE2b-256 824e76492339319c7d2b52961fe7efb2b036015d54c315366396def69db3cde3

See more details on using hashes here.

File details

Details for the file gce_rescue-2.0.0-py3-none-any.whl.

File metadata

  • Download URL: gce_rescue-2.0.0-py3-none-any.whl
  • Upload date:
  • Size: 238.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.7

File hashes

Hashes for gce_rescue-2.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 4ef02a79b07d958d9168a89b360eb99f37d8770f5883189334171e6ba4ba5ace
MD5 712466c82f74d4d0f3d4ca8a18adebe1
BLAKE2b-256 ffcced436d5a371bd75a2a161ce3b295be4f807e13475e6a657d48dfc3dcf64c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page