Skip to main content

A lightweight folder-based job queue system

Project description

QueueDir

A lightweight, cross-platform folder-based job queue system. Monitors an input folder and executes a script for each file that appears.

Installation

pip install queuedir

Usage

queuedir --watch /path/to/inbox --script /path/to/process.sh

Options

Option Short Description Default
--watch -w Folder to monitor (required) -
--script -s Script to execute (required) -
--done-dir Destination for processed files {watch}/done
--failed-dir Destination for failed files {watch}/failed
--timeout Script timeout in seconds 1200
--poll-interval File stability check interval 2
--once Process existing files and exit -
--verbose -v Enable debug logging -

Example

# Monitor inbox, run process.py for each file
queuedir -w ./inbox -s ./process.py -v

# Process existing files once and exit
queuedir -w ./inbox -s ./process.sh --once

How It Works

  1. Files dropped into the watched folder are detected
  2. QueueDir waits for the file to stabilize (no longer being written)
  3. The configured script is executed with the file path as argument
  4. On success (exit 0): file moves to done/
  5. On failure (non-zero exit): file moves to failed/

Script Requirements

Your script receives the file path as the first argument:

#!/bin/bash
filepath="$1"
echo "Processing: $filepath"
# ... do work ...
exit 0  # success
#!/usr/bin/env python
import sys
filepath = sys.argv[1]
print(f"Processing: {filepath}")
# ... do work ...
sys.exit(0)  # success

License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

queuedir-1.0.0.tar.gz (9.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

queuedir-1.0.0-py3-none-any.whl (8.3 kB view details)

Uploaded Python 3

File details

Details for the file queuedir-1.0.0.tar.gz.

File metadata

  • Download URL: queuedir-1.0.0.tar.gz
  • Upload date:
  • Size: 9.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.2

File hashes

Hashes for queuedir-1.0.0.tar.gz
Algorithm Hash digest
SHA256 efd23de4bc0c92d2d7284b7c8d23b9646b2a3f8c8c627ced1aa9797ce8baee09
MD5 e7ded0739fc60d3c67eb273e6bac1bf9
BLAKE2b-256 9f63118e086cb98da896dfa99e49c8895ee89b6f6b79215ef671846987072868

See more details on using hashes here.

File details

Details for the file queuedir-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: queuedir-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 8.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.2

File hashes

Hashes for queuedir-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 2ab3fbeebc2edcf01402d737e17175c3b8dc0c7338aa8ea7778b822ac306e5e1
MD5 daf4dec7ee18d42edb4098c1d7c46b93
BLAKE2b-256 76a16468c6702db24ac5d0ff012194d2edb48d32ba68dacda75cdfe9db103f37

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page