Skip to main content

User-friendly Postfix queue data filter

Project description

PostQF

Copyright © 2022 Ralph Seichter

PostQF is a user-friendly Postfix queue data filter which operates on data produced by postqueue -j. See the manual page's subsection titled "JSON object format" for details. PostQF offers convenient features for analysis and and cleanup of Postfix mail queues.

I have used the all-purpose JSON manipulation utility "jq" before, but found it inconvenient for everyday Postfix administration tasks. "jq" offers great flexibility and handles all sorts of JSON input, but it comes at the cost of complexity. PostQF is an alternative specifically tailored for easier access to Postfix queues.

To facilitate the use of Unix-like pipelines, PostQF usually reads from stdin and writes to stdout. Using command line arguments, you can override this behaviour and define one or more input files and/or an output file. Depending on the context, a horizontal dash - represents either stdin or stdout. See the command line usage description below.

Example usage

Find all messages in the deferred queue where the delay reason contains the string connection timed out.

postqueue -j | postqf -q deferred -d 'connection timed out'

Find all messages in the active or hold queues which have at least one recipient in the example.com or example.org domains, and write the matching JSON records into the file /tmp/output.

postqueue -j | postqf -q 'active|hold' -r '@example\.(com|org)' -o /tmp/output

Find all messages all queues for which the sender address is alice@gmail.com or bob@gmail.com, and pipe the queue IDs to postsuper in order to place the matching messages on hold.

postqueue -j | postqf -s '^(alice|bob)@gmail\.com$' -i | postsuper -h -

Print the number of messages which arrived during the last 30 minutes.

postqueue -j | postqf -a 30m | wc -l

The final example assumes a directory /tmp/data with several files, each containing JSON output produced at some previous time. The command pipes all queue IDs which have ever been in the hold queue into the file idlist, relying on BASH wildcard expansion to generate a list of input files.

postqf -i -q hold /tmp/data/*.json > idlist

Filters

Queue entries can be easily filtered by

  • Arrival time
  • Delay reason
  • Queue name
  • Recipient address
  • Sender address

and combinations thereof, using regular expressions. Anchoring is optional, meaning that plain text is treated as a substring pattern.

Time based filters

Arrival time filters do not use regular expressions, but support the following formats instead:

  1. ISO 8601 time strings.
  2. Unix time (the number of seconds since January 1, 1970). This is the representation of arrival time returned in JSON-format Postfix queue data.
  3. Time difference, expressed as one or more digits followed by a single "unit" character s, m, h, or d. These units designate seconds, minutes, hours and or days. The resulting timestamp will be in the past, as in "now minus the difference".

Please keep in mind that formats 1 and 2 are used for fixed timestamps, while format 3 represents time differences against the time of running PostQF. When format 3 is used with static input data (say, JSON data you saved to disk sevaral days ago) the results may vary as time progresses. When in doubt, use absolute time formats.

The command line option -a X means "message arrived after time X", and -b Y means "message arrived before time Y". The filter string can have any of the supported formats, and you can mix them freely. Here are some examples of valid command line arguments:

  • -a 2022-01-23T08:30 -b 2022-01-23T17:45 January 23, 2022 between 08:30 and 17:45.
  • -a 1642923000 -b 1642956300 The same time interval, specified in Unix time.
  • -a 90m Less than 90 minutes ago.
  • -b 36h More than 36 hours days ago.

Command line usage

postqf [-h] [-i] [-d [REGEX]] [-q [REGEX]] [-r [REGEX]] [-s [REGEX]]
       [-a [TS]] [-b [TS]] [-o [PATH]] [PATH [PATH ...]]

positional arguments:
  PATH        Input file. Use a dash "-" for standard input.

optional arguments:
  -h, --help  show this help message and exit
  -i          ID output only.
  -o [PATH]   Output file. Use a dash "-" for standard output.

Regular expression filters:
  -d [REGEX]  Delay reason filter.
  -q [REGEX]  Queue name filter.
  -r [REGEX]  Recipient address filter.
  -s [REGEX]  Sender address filter.

Arrival time filters:
  -a [TS]     Message arrived after TS.
  -b [TS]     Message arrived before TS.

Installation

The only installation requirement is Python 3.7 or newer. PostQF is distributed via PyPI.org and can usually be installed using pip. If this fails, or if both Python 2.x and 3.x are installed on your machine, use pip3 instead.

If possible, use the recommended installation with a Python virtual environment. Site-wide installation usually requires root privileges.

# Recommended: Installation using a Python virtual environment.
mkdir ~/postqf
cd ~/postqf
python3 -m venv .venv
source .venv/bin/activate
pip install -U pip postqf
# You can now execute PostQF. The following displays helpful information:
postqf -h
# After logging in afresh in the future, you need only activate the venv again:
source ~/postqf/.venv/bin/activate
postqf -h
# Alternative method: Site-wide installation, requires root access.
sudo pip install postqf

The pip installation process adds a launcher executable postqf, either site-wide or in the Python virtual environment. In the latter case, the launcher will be placed into the directory .venv/bin which is automatically added to your PATH variable when you activate the venv environment as shown above.

Contact

The project is hosted on GitHub in the rseichter/postqf repository. If you have suggestions or run into any problems, please check the discussions section first. There is also an issue tracker available, and the build configuration file contains a contact email address.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

postqf-0.2.tar.gz (20.3 kB view hashes)

Uploaded Source

Built Distribution

postqf-0.2-py3-none-any.whl (25.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page