flume-engine 1.0.1

Rule-based file operations engine for Linux

Flume

A rule-based file operations engine for Linux. Define rules in YAML -- when files matching patterns appear in watched directories, Flume executes actions: move, copy, rename, convert, sync, archive, deduplicate.

Think Hazel for macOS, but CLI-first, config-driven, no AI, no cloud dependency.

Features

• Watch directories for new files/folders via inotify
• Pattern matching with glob and regex support
• Size filtering -- match files by min/max size
• Actions: move, copy, rename, trash, run shell commands, unzip, desktop notifications
• Rule chaining -- output of one action feeds the next
• One-shot or daemon mode -- run once or watch continuously
• Dry-run mode -- preview what would happen without touching anything
• Summary reports -- export dry-run results as text, JSON, or CSV
• Structured logging -- every operation logged with timestamps
• Config validation -- catch errors before they bite

Installation

pip install flume-engine

Or install from source:

git clone https://github.com/mattWoolly/Flume.git
cd Flume
pip install .

Quick Start

Create ~/.config/flume/rules.yaml:

rules:
  - name: organize-downloads
    watch: ~/Downloads
    match:
      pattern: "*.pdf"
    actions:
      - move: ~/Documents/PDFs/

Then run:

# Preview what would happen
flume run --dry-run

# Run once against existing files
flume run

# Or watch continuously for new files
flume watch

Configuration

Rules live in ~/.config/flume/rules.yaml:

rules:
  - name: archive-zips
    watch: ~/Downloads
    match:
      pattern: "*.zip"
    actions:
      - unzip:
          dest: ~/Documents/extracted/
      - trash-original: true

  - name: organize-screenshots
    watch: ~/Pictures/Screenshots
    match:
      pattern: "*.png"
    actions:
      - rename: "{date:%Y-%m-%d}_{original}"
      - move: ~/Pictures/Screenshots/organized/

  - name: large-file-alert
    watch: ~/Downloads
    match:
      pattern: "*"
      min_size: 1GB
    actions:
      - notify:
          title: "Large download"
          body: "File arrived: {name}"
          urgency: critical

  - name: cleanup-old-logs
    watch: /var/log/myapp
    match:
      pattern: "*.log"
      older_than: 30
    actions:
      - compress:
          dest: /var/log/myapp/archive/
      - trash-original: true

Running as a Service

Flume ships a systemd user unit in contrib/flume.service. To install it:

# Copy the unit file
mkdir -p ~/.config/systemd/user
cp contrib/flume.service ~/.config/systemd/user/

# Enable and start
systemctl --user daemon-reload
systemctl --user enable flume
systemctl --user start flume

# Check status
systemctl --user status flume

# View logs
journalctl --user -u flume -f

This runs flume watch under your user session. It restarts automatically on failure and starts on login. Make sure flume is installed to ~/.local/bin (i.e. pip install --user -e . or pip install -e . in a virtualenv on your PATH).

To keep it running after logout (on a server, for example):

loginctl enable-linger $USER

Architecture

flume/
  cli.py          # CLI entry point (click)
  config.py       # YAML config loading and validation
  watcher.py      # inotify filesystem watcher
  matcher.py      # Pattern/rule matching engine
  actions/        # Action implementations
    move.py
    copy.py
    rename.py
    trash.py
    unzip.py
    shell.py
    notify.py
  engine.py       # Rule execution engine
  log.py          # Structured logging

Requirements

• Python 3.10+
• Linux (inotify-based; macOS/BSD support planned)

License

MIT