inferyx-monitoring 1.0.33

Monitor batch pipelines via API and email alerts — install and deploy on Linux servers from PyPI

inferyx-monitoring

Monitor batch jobs from a CSV file, poll the Inferyx API, and send email (and optional Teams / Google Chat) alerts.

Version: 1.0.33 · PyPI: inferyx-monitoring · CLI: inferyx-monitoring

This page is the full guide: Install, Upgrade, and Whats new. Everything is on this one page — no external doc links.

---

Contents

• Install
• Upgrade
• Admin UI
• Whats new

---

Install

First-time setup on a Linux server. Default paths use /opt/pipeline-monitor and user inferyx.

Quick install (script)

Copy both scripts to the server (from git repo server/), or use them from pip after install:

sudo bash install_inferyx_monitoring.sh

With Admin UI:

sudo bash install_inferyx_monitoring.sh --admin --admin-ui --install-nginx

Pin a version:

sudo bash ./install_inferyx_monitoring.sh --pin 1.0.32

After pip install, scripts are also at (run with bash or chmod +x first):

/opt/pipeline-monitor/.venv/share/inferyx-monitoring/scripts/

sudo bash /opt/pipeline-monitor/.venv/share/inferyx-monitoring/scripts/install_inferyx_monitoring.sh

The script creates the user, venv, installs from PyPI, runs --init-config, and enables systemd.

Manual install (step by step)

Step 1 — Python and package

sudo apt update && sudo apt install -y python3 python3-venv python3-pip
id inferyx || sudo useradd --system --home-dir /opt/pipeline-monitor --shell /usr/sbin/nologin inferyx
sudo mkdir -p /opt/pipeline-monitor && sudo chown inferyx:inferyx /opt/pipeline-monitor
sudo -u inferyx python3 -m venv /opt/pipeline-monitor/.venv
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install --upgrade pip inferyx-monitoring

Step 2 — Config files

sudo -u inferyx /opt/pipeline-monitor/.venv/bin/inferyx-monitoring --init-config --work-dir /opt/pipeline-monitor

Creates /opt/pipeline-monitor/.env and /opt/pipeline-monitor/batch_file.csv if missing. On every start the monitor adds missing keys only — it does not overwrite SMTP/API secrets or batch rows.

Step 3 — Edit .env

sudo -u inferyx vi /opt/pipeline-monitor/.env
sudo chmod 600 /opt/pipeline-monitor/.env

Required variables

Variable	Description
PIPELINE_SMTP_HOST, PIPELINE_SMTP_PORT, PIPELINE_SMTP_USERNAME, PIPELINE_SMTP_PASSWORD	SMTP
PIPELINE_FROM_NAME	Sender name
PIPELINE_MAIL_TO	Alert recipients
PIPELINE_API_BASE_URL	API base URL (no name= in the URL)
PIPELINE_API_TOKEN, PIPELINE_API_TOKEN_HEADER	API auth
PIPELINE_DEVOPS_EMAIL	Recipient for no_data alerts

Recommended scheduling

PIPELINE_CHECK_MODE=schedule_windows
PIPELINE_CHECK_WINDOW_MINUTES=10
PIPELINE_SCHEDULE_GRACE_MINUTES=5
PIPELINE_API_FILTER_BY_SCHEDULE_DATE=false

Step 4 — Edit batch_file.csv

One batch per row. Last column: Active or Suspended. Use 24-hour times.

Name,Frequency,ExpectedStartTime,AvgExecutionTime,ExpectedDayOfMonth,Status
my_batch,Daily,9:00:00,"10 mins",,Active

Step 5 — Test run

Run as one line (do not split with \):

sudo -u inferyx /opt/pipeline-monitor/.venv/bin/inferyx-monitoring --once --work-dir /opt/pipeline-monitor --env-file /opt/pipeline-monitor/.env --csv-file /opt/pipeline-monitor/batch_file.csv

Step 6 — systemd service

sudo tee /etc/systemd/system/inferyx-monitoring.service <<'EOF'
[Unit]
Description=Inferyx Pipeline Batch Monitor
After=network-online.target

[Service]
Type=simple
User=inferyx
Group=inferyx
WorkingDirectory=/opt/pipeline-monitor
ExecStart=/opt/pipeline-monitor/.venv/bin/inferyx-monitoring --work-dir /opt/pipeline-monitor --env-file /opt/pipeline-monitor/.env --csv-file /opt/pipeline-monitor/batch_file.csv
Restart=always
RestartSec=10
Environment=PYTHONUNBUFFERED=1

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now inferyx-monitoring.service

Paths reference

Item	Path
Install directory	/opt/pipeline-monitor
Config	/opt/pipeline-monitor/.env
Batch list	/opt/pipeline-monitor/batch_file.csv
Log	/opt/pipeline-monitor/pipeline_script.log
Python venv	/opt/pipeline-monitor/.venv
Service user	inferyx
systemd unit	inferyx-monitoring.service

---

Upgrade

Latest version: 1.0.33

Quick upgrade (script)

sudo bash ./upgrade_inferyx_monitoring.sh
sudo bash ./upgrade_inferyx_monitoring.sh --mode admin
sudo bash ./upgrade_inferyx_monitoring.sh --mode full
sudo bash ./upgrade_inferyx_monitoring.sh --mode ui
sudo bash ./upgrade_inferyx_monitoring.sh --mode full --pin 1.0.32

From an existing install:

sudo bash /opt/pipeline-monitor/.venv/share/inferyx-monitoring/scripts/upgrade_inferyx_monitoring.sh --mode full

Mode	What it does
monitor	pip install --upgrade inferyx-monitoring + restart monitor
admin	Upgrade [admin] + restart monitor and admin API
full	Upgrade [admin] + refresh UI + restart all + reload nginx
ui	Refresh Admin UI static files only

Legacy CSV rename (from 1.0.14 or older): add --legacy-migrate.

Manual upgrade

Pick the row that matches what you want to update, run the commands, then restart the service.

Monitor only

sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install --upgrade inferyx-monitoring
sudo systemctl restart inferyx-monitoring.service

Monitor + Admin API

sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install --upgrade 'inferyx-monitoring[admin]'
sudo systemctl restart inferyx-monitoring.service
sudo systemctl restart inferyx-monitoring-admin.service

Full stack (monitor + admin API + admin UI)

sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install --upgrade 'inferyx-monitoring[admin]'
sudo inferyx-monitoring-admin-install-ui --target /var/www/pipeline-monitor-admin
sudo systemctl restart inferyx-monitoring.service
sudo systemctl restart inferyx-monitoring-admin.service
sudo systemctl reload nginx

Admin UI files only

Use this when the Python package is already up to date and you only need new static UI files.

From pip (after upgrading the package):

sudo inferyx-monitoring-admin-install-ui --target /var/www/pipeline-monitor-admin
sudo systemctl reload nginx

From deploy tarball (copy inferyx-monitoring-deploy-VERSION.tar.gz to the server):

tar -xzf /tmp/inferyx-monitoring-deploy-VERSION.tar.gz -C /tmp
cd /tmp/inferyx-monitoring-deploy-VERSION
sudo ./scripts/install_admin_ui_deploy.sh
sudo systemctl reload nginx

Upgrade from 1.0.14 or older

sudo -u inferyx mv /opt/pipeline-monitor/jfl_batch.csv /opt/pipeline-monitor/batch_file.csv
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install --upgrade inferyx-monitoring
sudo systemctl restart inferyx-monitoring.service

Add to .env if missing:

PIPELINE_CHECK_MODE=schedule_windows
PIPELINE_CHECK_WINDOW_MINUTES=10

Update systemd ExecStart to use --csv-file /opt/pipeline-monitor/batch_file.csv if it still points at jfl_batch.csv.

Common problems after upgrade

Problem	Fix
unrecognized arguments:	Run CLI commands as one line
No email	Check SMTP settings in .env
API errors	Remove name= from API URL; set PIPELINE_API_FILTER_BY_SCHEDULE_DATE=false
Service crash on 1.0.25	Upgrade to 1.0.26 or newer
Alert crash NameError: signature	Upgrade to 1.0.21 or newer
inferyx-monitoring-admin --help starts server	Upgrade to 1.0.28 or newer

---

Admin UI

Optional web UI to edit .env and batch CSV. Requires OAuth. Servers do not need Node.js or npm — UI files are pre-built in the pip package.

First-time Admin UI setup

1. Install admin extra

sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install 'inferyx-monitoring[admin]'

2. Auth policy

sudo mkdir -p /etc/pipeline-monitor
sudo cp /opt/pipeline-monitor/.venv/share/inferyx-monitoring/config/auth.policy.example /etc/pipeline-monitor/auth.policy
sudo chmod 600 /etc/pipeline-monitor/auth.policy
sudo vi /etc/pipeline-monitor/auth.policy

Set OAuth providers and ui.enabled: true.

3. Install UI static files

sudo inferyx-monitoring-admin-install-ui --target /var/www/pipeline-monitor-admin

4. Admin API service

sudo tee /etc/systemd/system/inferyx-monitoring-admin.service <<'EOF'
[Unit]
Description=Inferyx Pipeline Monitor Admin API
After=network-online.target

[Service]
Type=simple
User=inferyx
Group=inferyx
WorkingDirectory=/opt/pipeline-monitor
ExecStart=/opt/pipeline-monitor/.venv/bin/inferyx-monitoring-admin --host 127.0.0.1 --port 8090
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now inferyx-monitoring-admin.service

5. Nginx

Proxy /api/ to 127.0.0.1:8090 and serve static files from /var/www/pipeline-monitor-admin at /admin/.

Example snippet (adjust server name and TLS):

location /api/ {
    proxy_pass http://127.0.0.1:8090/;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
}

location /admin/ {
    alias /var/www/pipeline-monitor-admin/;
    try_files $uri $uri/ /admin/index.html;
}

Copy the full example:

sudo cp /opt/pipeline-monitor/.venv/share/inferyx-monitoring/config/nginx-pipeline-monitor-admin.conf.example /etc/nginx/sites-available/pipeline-monitor-admin
sudo ln -sf /etc/nginx/sites-available/pipeline-monitor-admin /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

Admin CLI

Use the venv path (sudo does not see the venv automatically):

/opt/pipeline-monitor/.venv/bin/inferyx-monitoring-admin --help
/opt/pipeline-monitor/.venv/bin/inferyx-monitoring-admin-install-ui --target /var/www/pipeline-monitor-admin

---

Whats new

1.0.33

• Fix inferyx-monitoring-admin-install-ui — was looking in wrong directory for bundled UI files

Upgrade: pip install --upgrade 'inferyx-monitoring[admin]==1.0.33' then run install-ui again.

1.0.32

• Server install script: install_inferyx_monitoring.sh
• Server upgrade script: upgrade_inferyx_monitoring.sh (modes: monitor, admin, full, ui)
• Scripts ship in pip package under share/inferyx-monitoring/scripts/

Upgrade: sudo ./upgrade_inferyx_monitoring.sh --mode full or see Upgrade above.

1.0.31

• Single-page PyPI documentation: install, upgrade, and whats new on this README only
• Pip package no longer bundles separate doc files

Upgrade: pip install --upgrade inferyx-monitoring then restart services (see Upgrade above).

1.0.30

• Pre-built Admin UI in pip wheel — no npm on production servers
• inferyx-monitoring-admin-install-ui command
• Deploy tarball inferyx-monitoring-deploy-VERSION.tar.gz for UI-only updates

Upgrade: pip install --upgrade 'inferyx-monitoring[admin]' then run inferyx-monitoring-admin-install-ui if you use the Admin UI.

1.0.29

• Release notes moved to this README (PyPI cannot host separate doc files)

1.0.28

• inferyx-monitoring-admin --help and --version work without starting the server
• Nginx and systemd examples for Admin UI

1.0.27

• Admin web UI for .env and batch CSV (OAuth, audit log)
• Safer login error messages; plain-text mail status (no icons)

1.0.26

• Hotfix for startup crash on 1.0.25 (CHAT_CONFIG import error) — upgrade immediately if on 1.0.25

1.0.25

• Microsoft Teams and Google Chat webhook alerts
• Test with: inferyx-monitoring --test-chat-alerts --work-dir /opt/pipeline-monitor

1.0.21

• Fix alert email crash (NameError: signature)

1.0.20

• Auto-migration on start: adds missing .env keys, migrates jfl_batch.csv to batch_file.csv

1.0.15

• Default batch file renamed to batch_file.csv
• --init-config command
• PIPELINE_CHECK_MODE=schedule_windows recommended