snakemake-logger-plugin-flowo 0.1.11

A unified PostgreSQL-based logging plugin and backend for Snakemake

Flowo 🚀

Welcome to Flowo – your real-time Snakemake workflow dashboard!

Unleash the power of automation and monitoring with a fresh, interactive experience.

Demo page: flowo online

---

✨ Features

• ⚡ Real-time Monitoring:
  Flowo leverages Server-Sent Events (SSE) and PostgreSQL's LISTEN/NOTIFY mechanism for instant workflow updates. The frontend refreshes automatically – just sit back and watch the magic happen!

• 🚀 Efficient Data Fetching:
  With smart polling (every 5 seconds) and debouncing, Flowo keeps your data up-to-date without overloading the network. Fast, efficient, and always fresh.

• 🔍 Search & Filtering:
  Instantly find workflows by name, tags, or user. Delete workflows from the database with a click (no worries, your files are safe).

• 🧩 Rule-based Filtering:
  Focus on specific rules to filter jobs and update the execution timeline. Debug and optimize like a pro!

• 🖼️ Result Preview:
  Instantly preview output files generated by your favorite rules, right in the Results submenu. No more hunting for results!

• 📋 Logging System:
  Dive into detailed workflow and job logs. Analyze, learn, and improve your processes with ease.

---

🛠️ Workflows

📊 Dashboard

🕹️ Jobs

---

🚦 Installation

• Requirements:
  • 🐧 Linux system
  • 🐳 Docker
  • 🐍 snakemake >= 9.6.2

Flowo consists of two parts:

• The Flowo web service (this repository)
• The Snakemake plugin: snakemake-logger-plugin-flowo

---

1️⃣ Install the Flowo Web Service

git clone https://github.com/zhanghaomiao/flowo.git
cd flowo
cp env.example .env

2. Build and Run Backend:

docker compose up -d --build

Edit your .env file as follows. In most cases, you only need to modify ⚠️ FLOWO_WORKING_PATH to point to the directory where you run your Snakemake workflows:

# Application Settings
DOMAIN=localhost
PORT=3100
TZ=Asia/Shanghai

# Database Configuration
POSTGRES_DB=flowo_logs
POSTGRES_USER=flowo
POSTGRES_PASSWORD=flowo_password
POSTGRES_HOST=localhost
POSTGRES_PORT=5432

# Workflow Directory
FLOWO_WORKING_PATH=/path/to/flowo_project_dir  # ⚠️ Important: set this to your workflow directory

Start the Flowo web service:

docker compose -f docker-compose.yml up -d

Open http://localhost:3100 in your browser.
If you see the Flowo interface and the "Live Updates" icon is green, congratulations—your Flowo web service is running successfully! 🎉

---

2️⃣ Install the Snakemake Logger Plugin

Next, let's install the Snakemake plugin.
For detailed installation, configuration, and usage, please refer to the plugin's documentation.

Quick start:

pip install .  # Install the plugin from the root of this repo
flowo-init-config --generate-config

Configure the plugin:

vim $HOME/.config/flowo/.env

Edit your $HOME/.config/flowo/.env as follows:

### Postgres settings
POSTGRES_USER=flowo         # same as your .env
POSTGRES_PASSWORD=flowo_password  # same as your .env
POSTGRES_DB=flowo_logs      # same as your .env
POSTGRES_HOST=localhost     # same as your .env
POSTGRES_PORT=5432          # same as your .env

### APP settings
# FLOWO_USER has been removed.
# Authentication is handled via FLOWO_USER_TOKEN.
FLOWO_USER_TOKEN=your_generated_token_here
FLOWO_WORKING_PATH=/path/to/flowo_project_dir # ⚠️ same as your .env

---

🚀 Usage

All set! 🎉
Let's test with a demo project:

cd /path/to/flowo_project_dir
mkdir demo                # Create a demo project folder
cd demo
wget https://raw.githubusercontent.com/zhanghaomiao/flowo/refs/heads/main/tests/demos/Snakefile
# Run snakemake with flowo logger
snakemake \
    --logger flowo \
    --logger-flowo-name=your_project_name \
    --logger-flowo-tags="tagA,tagB,tagC"

Now, check out your workflow in the Flowo web! Good luck! 🍀