snakemake-logger-plugin-flowo 1.0.4

A unified PostgreSQL-based logging plugin and backend for Snakemake

Flowo 🚀

Welcome to Flowo – your real-time Snakemake workflow dashboard!

Unleash the power of automation and monitoring with a fresh, interactive experience.

Demo page: flowo online Documentation: flowo-docs

---

📰 News

Latest Release: Flowo now supports user authentication and login! Additionally, we've simplified the configuration process, making it even easier to get started with workflow monitoring and management.

---

✨ Features

• ⚡ Real-time Monitoring: Flowo leverages Server-Sent Events (SSE) and PostgreSQL's LISTEN/NOTIFY mechanism for instant workflow updates. The frontend refreshes automatically – just sit back and watch the magic happen!

• 🚀 Efficient Data Fetching: With smart polling (every 5 seconds) and debouncing, Flowo keeps your data up-to-date without overloading the network. Fast, efficient, and always fresh.

• 🔍 Search & Filtering: Instantly find workflows by name, tags, or user. Delete workflows from the database with a click (no worries, your files are safe).

• 🧩 Rule-based Filtering: Focus on specific rules to filter jobs and update the execution timeline. Debug and optimize like a pro!

• 🖼️ Result Preview: Instantly preview output files generated by your favorite rules, right in the Results submenu. No more hunting for results!

• 📋 Logging System: Dive into detailed workflow and job logs. Analyze, learn, and improve your processes with ease.

---

🛠️ Workflows

📊 Dashboard

🕹️ Jobs

---

🚦 Installation

• Requirements:
  • 🐧 Linux system
  • 🐳 Docker
  • 🐍 snakemake

---

1️⃣ Install the Snakemake Logger Plugin

Now let’s install snakemake-logger-plugin-flowo, a logging plugin for Snakemake. Once the installation is complete, simply add --logger flowo to your Snakemake command, and the execution status of your workflow will be automatically sent to the Flowo web server for real-time monitoring and visualization.

pip install snakemake-logger-plugin-flowo

2️⃣ Install the Flowo web server

git clone https://github.com/zhanghaomiao/flowo.git
cd flowo
cp env.example .env

Edit your .env file as follows. In most cases, you only need to modify FLOWO_WORKING_PATH to point to the directory where you run your Snakemake workflows. If you don’t need to view Snakefiles, log files, result previews, etc. in the Flowo web interface — and only want to monitor workflow, job, and rule execution status — there is no need to configure these additional paths.

# Application Settings
DOMAIN=localhost
PORT=3100
TZ=Asia/Shanghai

# Database Configuration
POSTGRES_DB=flowo_logs
POSTGRES_USER=flowo
POSTGRES_PASSWORD=flowo_password
POSTGRES_HOST=localhost
BACKEND_CORS_ORIGINS=["*"]

# Workflow Directory
FLOWO_WORKING_PATH=/path/to/flowo_project_dir  #Important: set this to your workflow directory

Start the Flowo web service:

docker compose up -d

Open http://localhost:3100 in your browser. Create your account and log in. If you see the Flowo interface and the "Live Updates" icon is green, congratulations—your Flowo web service is running successfully!

Next, create a token, which is required to configure the snakemake-logger-plugin-flowo. Click the user icon in the top-right corner, select Generate New Token, then provide a name and set an expiration time. Once confirmed, the token will be generated successfully.

After that, click Config to access a CLI snippet. Copy and run this command in the terminal where snakemake-logger-plugin-flowo is installed. Once it finishes executing, the setup will be complete.

---

🚀 Usage

All set! Let's test with a demo project:

cd /path/to/flowo_project_dir
mkdir demo                # Create a demo project folder
cd demo
wget https://raw.githubusercontent.com/zhanghaomiao/flowo/refs/heads/main/tests/demos/Snakefile
# Run snakemake with flowo logger
snakemake \
    --logger flowo \
    --logger-flowo-name=your_project_name \
    --logger-flowo-tags="tagA,tagB,tagC"

Now, check out your workflow in the Flowo web! Good luck! 🎉

Developer Guide

📦 Docker Environments

We provide two ways to run Flowo using Docker:

1. Production Mode (docker-compose.yml): Uses pre-built images from GitHub Container Registry (GHCR). Ideal for deployment.

   docker compose pull
   docker compose up -d
   

2. Development Mode (docker-compose.dev.yml): Builds images from your local source code. Ideal for testing changes.

   docker compose -f docker-compose.dev.yml up --build
   

🏷️ Versioning & Releases

Versioning is handled automatically through GitHub Actions.

Automating Version Bumps

To increment the project version:

1. Go to the Actions tab in your GitHub repository.
2. Select the "Bump Version" workflow.
3. Click "Run workflow", choose the bump type (patch, minor, or major), and run.
4. This will update pyproject.toml and commit the change automatically.

Creating Releases

Whenever you push to the release branch, a new GitHub Release is created, and Docker images are pushed to GHCR with the version tag.

---