Test-on-Colab

Programmatic toolkit facilitating the testing of locally-developed Jupyter notebooks for transfer to Google Colab

Features

• Uses Google's official Colab Docker runtime image
• Executes notebooks programmatically with error handling
• Minimal output by default, verbose mode available
• Supports environment variables from .env files
• Intuitive CLI interface
• Docker cleanup utilities

Installation

From GitHub via pip

pip install git+https://github.com/ColinConwell/Test-On-Colab.git

From Source (Development)

# Clone the repository
git clone https://github.com/ColinConwell/Test-On-Colab.git test-on-colab
cd test-on-colab

# Install package in editable mode
pip install -e .

Quick Start

Test the Example Notebook

# From the colab-test directory
colab-test examples/simple_test.ipynb

Basic Usage

# Test a notebook (minimal output)
colab-test notebook.ipynb

# Show internal progress logs
colab-test notebook.ipynb --verbose
# or
colab-test notebook.ipynb -v

# Show raw Docker output (for debugging)
colab-test notebook.ipynb --docker-logs

# Install packages before testing
colab-test notebook.ipynb -r requirements.txt       # From requirements file
colab-test notebook.ipynb -p numpy pandas           # Specific packages
colab-test notebook.ipynb -p "scikit-learn>=1.0"    # With version constraints

# Use environment variables from a file
colab-test notebook.ipynb --env .env

# Force update to latest Docker image
colab-test notebook.ipynb --pull

# Specify Docker platform (auto-detects by default)
colab-test notebook.ipynb --platform linux/amd64

# Combine options
colab-test notebook.ipynb -v -r requirements.txt --env .env

Prerequisites

• Python 3.8+; Docker installed and running
• Sufficient Docker storage for the Colab image (~4-5GB compressed, ~33GB uncompressed)
  • Important: This refers to Docker's internal storage limit, not your system disk space
  • See Docker Storage section below for details

How It Works

1. Docker Environment: Uses Google's official Colab runtime image (us-docker.pkg.dev/colab-images/public/runtime)
2. Image Caching: After the first pull, the image is cached locally and reused automatically
3. Notebook Execution: Executes your notebook in the default Colab environment (no automatic package installation)
4. Isolation: Each test runs in a fresh container
5. Error Handling: Clear error messages and tracebacks on failure

Package Installation

By default, colab-test runs notebooks in the standard Colab environment without installing additional packages. This ensures your notebook works as-is in Colab.

Best practice: Include !pip install commands in your notebook:

!pip install package-name

Optional package installation: For testing purposes, you can install packages before execution:

# From a requirements.txt file
colab-test notebook.ipynb -r requirements.txt

# Specific packages
colab-test notebook.ipynb -p numpy pandas matplotlib

# With version constraints
colab-test notebook.ipynb -p "numpy>=1.20" "pandas<2.0"

When to use optional installation:

• Testing notebooks that will have packages pre-installed in a custom environment
• Quick prototyping without modifying the notebook
• CI/CD pipelines where you want to separate package specs from notebook code

Recommendation: For notebooks intended for Google Colab, prefer including pip install commands in the notebook itself to ensure portability.

Image Caching Behavior

• First run: Downloads the Colab image (~4-5GB, can take several minutes)
• Subsequent runs: Uses cached image (instant startup)
• Updates: Use --pull flag to update to the latest Colab image
• Storage: Image persists between runs, containers are removed after execution

Output Behavior

Default Mode (no flags)

Minimal output - only shows final result:

• ✅ Success: [SUCCESS] Notebook executed successfully
• ❌ Failure: [ERROR] with error type, message, and full traceback

Verbose Mode (--verbose or -v)

Shows structured internal progress logs with clear steps:

[STEP] Checking Docker environment
[INFO] Using cached Docker image: us-docker.pkg.dev/colab-images/public/runtime:latest
[STEP] Preparing execution environment
[INFO] Using platform: linux/amd64
[INFO] Installing packages from requirements.txt...
[STEP] Executing notebook in Colab environment
[INFO] Notebook: example.ipynb
[SUCCESS] Notebook executed successfully

All output includes log level prefixes: [STEP], [INFO], [WARNING], [ERROR], [SUCCESS]

Package installation logs: When using -r or -p, verbose mode shows what's being installed before execution.

Raw Docker Logs (--docker-logs)

Shows complete, unfiltered Docker output for debugging. Useful when:

• Debugging container issues
• Inspecting detailed pip install output
• Troubleshooting platform/architecture problems

Note: Debugger warnings and pip progress bars are automatically suppressed for cleaner output in all modes.

Platform Support

The tool automatically detects the appropriate Docker platform for your system:

• macOS (Apple Silicon/Intel): Auto-detects and uses linux/amd64 (Colab's platform)
• Linux: Uses Docker's default platform detection
• Manual override: Use --platform flag to specify (e.g., --platform linux/arm64)

This eliminates platform mismatch warnings on ARM-based systems while maintaining compatibility with Colab's runtime environment.

Environment Variables

Create a .env file for API keys and configuration:

# .env
OPENAI_API_KEY=your_key_here
HF_TOKEN=your_token_here

Then use:

colab-test notebook.ipynb --env .env

Docker Management

The package includes utilities for managing Docker resources:

from colab_test.docker_utils import cleanup_docker, check_colab_image

# Check if Colab image is available
is_available, message = check_colab_image()
print(message)

# Clean up Docker resources
cleanup_docker(verbose=True)

Or use the included script:

bash scripts/docker_cleanup.sh

CI/CD Integration

Use in GitHub Actions:

- name: Test Notebooks
  run: |
    pip install colab-test
    colab-test notebooks/example.ipynb

Exit codes:

• 0: Success
• 1: Failure (notebook error or system error)

Docker Storage Management

Understanding Docker Storage Limits

Docker uses its own storage system that's independent of your system disk space:

• macOS/Windows: Docker Desktop runs in a VM with a virtual disk (default: 64GB)
• Linux: Docker stores data in /var/lib/docker/

The Colab runtime image is large (~4-5GB compressed, ~33GB uncompressed), so you may encounter storage errors even with plenty of system disk space available.

Common Error: "no space left on device"

If you see this error during image pull:

write /var/lib/docker/tmp/GetImageBlob...: no space left on device

This means Docker's internal storage is full, not your system disk.

Cleanup Solutions

Option 1: Quick cleanup (recommended)

docker system prune -a -f

This removes all unused images, containers, and build cache.

Option 2: Use included cleanup script

bash scripts/docker_cleanup.sh

This provides detailed information about what's being cleaned.

Option 3: Manual cleanup

# Remove stopped containers
docker container prune -f

# Remove unused images
docker image prune -a -f

# Remove unused volumes
docker volume prune -f

Checking Docker Storage

# See Docker disk usage
docker system df

# See detailed breakdown
docker system df -v

Increasing Docker Storage (macOS/Windows)

If you frequently work with large images:

1. Open Docker Desktop
2. Go to Settings → Resources → Disk image size
3. Increase the limit (e.g., to 100GB or more)
4. Click Apply & Restart

Increasing Docker Storage (Linux)

Docker uses your system disk directly. If /var/lib/docker/ is full, you may need to:

• Clean up using the commands above
• Move Docker's data directory to a larger partition
• Expand your root partition

Troubleshooting

Docker Not Found

Install Docker from https://docs.docker.com/get-docker/

Docker Daemon Not Running

• macOS/Windows: Start Docker Desktop
• Linux: sudo systemctl start docker

Initial Startup Time

The first run downloads the Colab image (~4-5GB). This depends on your internet speed but subsequent runs will be much faster as the image is cached.