kopi-docka 2.0.2

Robust cold backups for Docker environments using Kopia

Kopi-Docka

Cold backup solution for Docker environments using Kopia.

Kopi-Docka performs consistent backups of Docker containers by stopping them, creating snapshots of docker-compose files and volumes, then restarting services. Backups are encrypted and stored in Kopia repositories with configurable retention policies.

---

Restore Process

Interactive Restore Wizard

Start the restore wizard:

kopi-docka restore

The wizard guides you through these steps:

1. Select Backup Session

Lists available backup sessions grouped by time (5-minute tolerance):

📅 Available Backup Sessions:

1. 2025-01-15 02:00:45
   Units: mysql-stack, nginx-stack (2 units, 8 volumes)

2. 2025-01-14 02:00:12
   Units: mysql-stack, nginx-stack, redis (3 units, 10 volumes)

Select session (number or 'q' to quit): _

2. Select Backup Unit

If multiple units in session, choose which to restore:

📦 Units in this backup session:

1. mysql-stack (3 volumes) - 02:00:45
2. nginx-stack (2 volumes) - 02:00:48

Select unit to restore (number or 'q' to quit): _

3. Confirm Restore

Review what will be restored:

✅ Selected: mysql-stack from 2025-01-15 02:00:45

📝 This will guide you through restoring:
  - Recipe/configuration files
  - 3 volumes

Proceed with restore? (yes/no/q): _

4. Recipe Restoration

Automatically restores to temporary directory:

📂 Restoring recipes...
   ✓ docker-compose.yml
   ✓ mysql-db_inspect.json
   ✓ mysql-web_inspect.json

📁 Restored to: /tmp/kopia-restore-20250115-020045/recipes/mysql-stack/

5. Volume Restoration

For each volume, choose immediate or manual restore:

📦 Volume Restoration:

📁 Volume: mysql-data
📸 Snapshot: a3f5d892...

⚠️  Restore 'mysql-data' NOW? (yes/no/q):

If you choose 'yes':

• Stops affected containers
• Creates safety backup in /tmp
• Restores volume directly
• Restarts containers

If you choose 'no':

• Shows manual restore instructions
• Continues to next volume

6. Service Restart

After restore completes, start services:

cd /tmp/kopia-restore-20250115-020045/recipes/mysql-stack/
docker compose up -d

Restore Options

Restore specific snapshot:

# List all snapshots
kopi-docka list --snapshots

# Restore uses interactive wizard
kopi-docka restore

Restore to different server:

# 1. Install Kopi-Docka on new server
pipx install kopi-docka

# 2. Connect to existing repository
kopi-docka init
# Configure same repository_path and password

# 3. Restore
kopi-docka restore

Manual volume restore:

If you declined automatic restore or need to restore later:

# Stop containers using the volume
docker ps -q --filter 'volume=mysql-data' | xargs docker stop

# Create safety backup
docker run --rm -v mysql-data:/source -v /tmp:/backup \
  alpine tar czf /backup/mysql-data-backup.tar.gz -C /source .

# Restore from Kopia
kopia snapshot restore SNAPSHOT_ID /target/path

# Or use tar stream
kopia snapshot restore SNAPSHOT_ID --mode tar | \
  docker run --rm -i -v mysql-data:/target alpine \
  sh -c "rm -rf /target/* /target/.[!.]* 2>/dev/null || true; \
  tar -xpf - --numeric-owner --xattrs --acls -C /target"

# Restart containers
docker start CONTAINER_ID

Restore Safety Features

• Automatic safety backups - Creates /tmp/volume-backup-*.tar.gz before overwriting
• Quit anytime - Press 'q' at any prompt to cancel
• Manual instructions - Shows exact commands if you decline automatic restore
• Secret redaction warnings - Alerts if restored files contain ***REDACTED***
• Container management - Stops/starts containers automatically during volume restore

---

Features

• Cold backups for data consistency
• Docker Compose stack awareness
• AES-256 encryption via Kopia
• Multiple storage backends (S3, B2, Azure, GCS, SFTP, local)
• Disaster recovery bundles
• Systemd timer integration with journald logging
• Configurable retention policies
• Interactive restore wizard
• Automatic dependency installation
• Full Kopia CLI compatibility (use Kopia directly with Kopi-Docka's repository)

---

Installation

pipx install kopi-docka

# Install system dependencies (Docker, Kopia, tar, openssl)
sudo kopi-docka install-deps

Alternative with pip:

pip install kopi-docka
sudo kopi-docka install-deps

Verify Installation

kopi-docka --version
kopi-docka check

---

Requirements

• Linux (Debian/Ubuntu recommended, also Arch/Fedora)
• Docker Engine 20.10+
• Kopia CLI 0.10+ (tested with 0.10-0.18)
• Python 3.10+
• tar, openssl (usually pre-installed)

Verify versions:

docker --version
kopia --version
python3 --version

Automatic Dependency Installation

kopi-docka check                    # Show missing dependencies
sudo kopi-docka install-deps        # Install automatically
sudo kopi-docka install-deps --dry-run  # Preview changes

Supported distributions: Debian/Ubuntu (apt), RHEL/CentOS/Fedora (yum/dnf), Arch Linux (pacman + AUR).

---

Quick Start

# 1. Install and verify
pipx install kopi-docka
sudo kopi-docka install-deps

# 2. Create configuration
kopi-docka new-config
# Edit ~/.config/kopi-docka/config.conf:
#   - repository_path: backup destination
#   - password: change default password

# 3. Initialize repository
kopi-docka init
kopi-docka change-password

# 4. Verify setup
kopi-docka list --units
kopi-docka dry-run

# 5. Create backup
kopi-docka backup

# 6. Create disaster recovery bundle
kopi-docka disaster-recovery

---

Configuration

Config file locations (by precedence):

• /etc/kopi-docka.conf (system-wide)
• ~/.config/kopi-docka/config.conf (user-specific)

Kopi-Docka uses a dedicated Kopia profile (~/.config/kopia/repository-kopi-docka.config) and does not interfere with existing Kopia configurations.

Example configuration:

[kopia]
repository_path = b2://bucket-name/kopia
password = secure-password
compression = zstd
encryption = AES256-GCM-HMAC-SHA256

[backup]
parallel_workers = 4
stop_timeout = 30

[retention]
daily = 7
weekly = 4
monthly = 12
yearly = 5

[logging]
level = INFO              # DEBUG, INFO, WARNING, ERROR
file = /var/log/kopi-docka.log  # Optional file logging
max_size_mb = 100        # Log rotation size
backup_count = 5         # Keep 5 rotated logs

Storage Backends

Backblaze B2:

export B2_APPLICATION_KEY_ID="key-id"
export B2_APPLICATION_KEY="key-secret"

repository_path = b2://bucket-name/kopia

AWS S3:

export AWS_ACCESS_KEY_ID="key-id"
export AWS_SECRET_ACCESS_KEY="key-secret"

repository_path = s3://bucket-name/kopia

Local filesystem:

repository_path = /backup/kopia-repository

Additional backends: Azure Blob Storage, Google Cloud Storage, SFTP.

---

Automated Backups

Systemd Timer Setup

sudo kopi-docka write-units
sudo systemctl daemon-reload
sudo systemctl enable --now kopi-docka.timer

Check Status

systemctl list-timers kopi-docka.timer
systemctl status kopi-docka.timer
journalctl -u kopi-docka -f

Journald Integration

When running under systemd, Kopi-Docka automatically outputs structured logs to the systemd journal with searchable fields:

# View all Kopi-Docka logs
journalctl -u kopi-docka

# Follow logs in real-time
journalctl -u kopi-docka -f

# Filter by specific backup unit
journalctl -u kopi-docka UNIT=my-stack

# Show only errors
journalctl -u kopi-docka -p err

# Show logs from last backup run
journalctl -u kopi-docka --since "1 hour ago"

# Filter by operation
journalctl -u kopi-docka OPERATION=backup

# Show with metadata
journalctl -u kopi-docka -o json-pretty

Structured fields available:

• UNIT - Backup unit name
• OPERATION - backup, restore, etc.
• DURATION - Operation duration in seconds
• STATUS - SUCCESS, FAILED, PARTIAL
• MODULE - Python module name
• LEVEL - Log level

Enhanced journald support with python3-systemd:

sudo apt install python3-systemd  # Debian/Ubuntu
# or
sudo kopi-docka install-deps  # Installs automatically

Custom Schedule

sudo systemctl edit kopi-docka.timer

[Timer]
OnCalendar=*-*-* 03:00:00  # Daily at 03:00
Persistent=true

---

Disaster Recovery

Disaster recovery bundles contain repository connection info, encrypted credentials, configuration, and restore scripts.

Create Bundle

kopi-docka disaster-recovery

Automatic bundle updates after each backup:

[backup]
update_recovery_bundle = true
recovery_bundle_path = /backup/recovery
recovery_bundle_retention = 3

Restore from Bundle

On a new server:

# 1. Install Kopi-Docka
pipx install kopi-docka

# 2. Decrypt bundle
openssl enc -aes-256-cbc -d -pbkdf2 \
  -in bundle.tar.gz.enc -out bundle.tar.gz

# 3. Extract and reconnect
tar -xzf bundle.tar.gz
cd kopi-docka-recovery-*/
sudo ./recover.sh

# 4. Restore data
kopi-docka restore

# 5. Start services
cd /tmp/kopia-restore-*/recipes/
docker compose up -d

---

CLI Commands

Backup & Restore

Command	Description
list --units	Show discovered backup units
list --snapshots	Show all backups in repository
backup	Create cold backup of selected units
restore	Interactive restore wizard (step-by-step)
dry-run	Simulate backup without changes

Repository

Command	Description
init	Initialize Kopia repository
repo-status	Check repository connection
repo-maintenance	Run Kopia maintenance
change-password	Change repository password

Configuration

Command	Description
new-config	Create config template
show-config	Display config (secrets masked)
edit-config	Open config in editor
check	Verify system dependencies
install-deps	Install missing dependencies

Disaster Recovery

Command	Description
disaster-recovery	Create encrypted DR bundle

Service

Command	Description
write-units	Generate systemd files
daemon	Run as systemd service

---

Kopia Profile Isolation

Kopi-Docka uses a separate Kopia profile and does not interfere with existing Kopia installations:

# Existing Kopia backups (default profile)
~/.config/kopia/repository.config
kopia snapshot create /home/user/documents

# Kopi-Docka (separate profile)
~/.config/kopia/repository-kopi-docka.config
kopi-docka backup

Both configurations operate independently with separate repositories, schedules, and retention policies.

---

Direct Kopia Usage

Kopi-Docka is a wrapper around Kopia. You can use Kopia CLI directly with Kopi-Docka's repository by specifying the config file:

Kopia Config File Location

# Kopi-Docka uses this config:
~/.config/kopia/repository-kopi-docka.config

# Your personal Kopia config (if any):
~/.config/kopia/repository.config

Using Kopia with Kopi-Docka Repository

Always specify the config file:

# Set as alias for convenience
alias kopia-docka="kopia --config-file ~/.config/kopia/repository-kopi-docka.config"

# Or use full command
kopia --config-file ~/.config/kopia/repository-kopi-docka.config COMMAND

Common Kopia Operations

List snapshots:

kopia --config-file ~/.config/kopia/repository-kopi-docka.config snapshot list
kopia --config-file ~/.config/kopia/repository-kopi-docka.config snapshot list --all

Show snapshot details:

kopia --config-file ~/.config/kopia/repository-kopi-docka.config snapshot show SNAPSHOT_ID

Restore snapshot manually:

kopia --config-file ~/.config/kopia/repository-kopi-docka.config \
  snapshot restore SNAPSHOT_ID /restore/target/path

Repository maintenance:

kopia --config-file ~/.config/kopia/repository-kopi-docka.config \
  maintenance run --full

Check repository status:

kopia --config-file ~/.config/kopia/repository-kopi-docka.config repository status

Search snapshots by tag:

# Find all backups for specific unit
kopia --config-file ~/.config/kopia/repository-kopi-docka.config \
  snapshot list --tags=unit:mysql-stack

# Find all volume snapshots
kopia --config-file ~/.config/kopia/repository-kopi-docka.config \
  snapshot list --tags=type:volume

Mount repository as filesystem:

mkdir /tmp/kopia-mount
kopia --config-file ~/.config/kopia/repository-kopi-docka.config \
  mount all /tmp/kopia-mount
# Browse backups, then unmount:
kopia --config-file ~/.config/kopia/repository-kopi-docka.config \
  unmount /tmp/kopia-mount

Export snapshot to tar:

kopia --config-file ~/.config/kopia/repository-kopi-docka.config \
  snapshot restore SNAPSHOT_ID --mode tar > backup.tar

Important Notes

• Always use --config-file when working with Kopi-Docka's repository
• Without --config-file, Kopia uses your default config (different repository!)
• Kopi-Docka tags snapshots with: unit, type (recipe/volume), backup_id, timestamp
• You can create custom snapshots in the same repository if needed
• Kopia policies are set per-unit (e.g., recipes/mysql-stack)

---

Troubleshooting

Missing Dependencies

kopi-docka check
sudo kopi-docka install-deps
sudo kopi-docka install-deps --dry-run  # Preview only

After installation, logout and login for docker group membership to take effect.

No Backup Units Found

Check Docker status and socket access:

docker ps
sudo usermod -aG docker $USER  # Logout/login required

Invalid Repository Password

Repository exists with different password:

# Update config with correct password, then:
kopi-docka init

To reinitialize (deletes existing backups):

sudo mv /backup/kopia-repository /backup/kopia-repository.backup
kopi-docka init

Permission Issues

sudo mkdir -p /backup/kopia-repository
sudo chown $USER:$USER /backup/kopia-repository

Restore Issues

No restore points found:

# Verify repository connection
kopi-docka repo-status

# List snapshots manually
kopi-docka list --snapshots

Volume restore fails:

# Check Docker is running
docker ps

# Ensure volume exists
docker volume ls | grep VOLUME_NAME

# Check available disk space
df -h

Redacted secrets in restored files:

Files containing ***REDACTED*** need manual intervention. Check *_inspect.json files in restored recipes and update environment variables manually before starting containers.

---

Development

Setup

git clone https://github.com/TZERO78/kopi-docka.git
cd kopi-docka
pip install -e ".[dev]"

Tasks

make format       # Format code (Black)
make check-style  # Lint (flake8)
make test         # Run tests
make test-unit    # Unit tests only
make test-coverage # Coverage report

Code Style

• Formatter: Black
• Linter: flake8
• Type hints: Encouraged
• Docstrings: Google style

---

Contributing

Contributions are welcome. Please:

1. Fork the repository
2. Create a feature branch
3. Make changes and add tests
4. Run make format and make test
5. Submit a pull request

Report issues: GitHub Issues

---

License

MIT License - see LICENSE

Copyright (c) 2025 Markus F. (TZERO78)

---

Credits

• Kopia - Backup engine with encryption and deduplication
• Docker - Container platform
• Typer - CLI framework

Kopi-Docka is an independent project with no official affiliation to Docker Inc. or the Kopia project.

---

Links

• PyPI Package
• GitHub Repository
• Issue Tracker
• Discussions