nflaunch

nflaunch is a command-line tool designed to simplify the launching of Nextflow pipelines on cloud batch services. It abstracts complex cloud configurations into a unified interface, currently supporting Google Cloud Batch.

This tool is under active development and intended for users with experience managing cloud resources and Nextflow pipelines. Ensure you understand the configurations and implications of job submissions before using it in production environments.

Table of Contents

• Key Features
• Quick Start
• Documentation
• Usage Example
• License

Key Features

• GCP Batch Support - Automated volume mounting and job execution on Google Cloud
• Dry-Run Mode - Validate job configurations before submission
• Plugin System - Extensible architecture for pipeline-specific enhancements
• Local & Remote Pipelines - Support for both published (e.g. nf-core public pipelines) and local pipeline directories
• Resume Capability - Resume failed pipeline runs

Quick Start

Prerequisites

• Python 3.10+
• Google Cloud CLI installed and authenticated

Installation

pip install nflaunch

Verify installation:

nflaunch --help

For detailed installation instructions, see the Installation Guide.

GCP Setup

Configure required permissions and authentication:

gcloud auth login
gcloud auth application-default login

Enable the following APIs in your GCP project:

• Google Cloud Batch API (batch.googleapis.com)
• Google Cloud Storage API (storage.googleapis.com)

For complete GCP setup instructions, see the GCP Setup Guide.

Documentation

Comprehensive documentation is available in the docs/ directory:

Getting Started

• Installation Guide - Detailed installation instructions and setup
• Quickstart Guide - Step-by-step examples to run your first pipeline
• GCP Setup - Google Cloud permissions, authentication, and configuration

Reference

• CLI Reference - Complete command-line options and examples
• Cloud Resources - Understanding resources created in GCP
• Plugins - Plugin system overview and development guide

Usage Example

Run the nf-core/demo pipeline with dry-run validation:

nflaunch \
  --base-bucket my-nextflow-bucket \
  --pipeline-name nf-core/demo \
  --pipeline-version 1.0.2 \
  --params-file examples/params.yaml \
  --samplesheet examples/samplesheet.csv \
  --config-file examples/resources.config \
  --project-id my-gcp-project \
  --region europe-west4 \
  --service-account-email sa@my-project.iam.gserviceaccount.com \
  --network default \
  --subnetwork default \
  --labels '{"env": "test", "tool": "nflaunch"}' \
  --dry-run

Remove --dry-run to submit the job.

For more examples and detailed usage instructions, see the Quickstart Guide.

License

This project is distributed under the terms of the Apache License 2.0.

Documentation: docs/ | Examples: examples/ | Changelog: CHANGELOG.md