dqrxfer

Utility to transfer Data Quality Report (DQR) results from remote sites to the central site.

Overview

dqrxfer is a client-side package used by DQRbuild nodes (e.g., LLO, LHO, Virgo, KAGRA, etc.) to push Data Quality Report files to a central DQRbuild node. It supports both HTTPS (with Shibboleth/ECP authentication) and SCP/SSH transfer methods.

Requirements

• A valid Kerberos ticket (TGT) or keytab for authentication
• Network access to the DQRbuild node

Installation

pip install dqrxfer

Commands

The package provides three command-line tools:

dqr-ping

Tests connectivity and authentication with the DQR server. Useful for verifying your Kerberos credentials and network access before attempting an upload.

usage: dqr-ping [-h] [-v] [-V] [-q] [--server SERVER] [--user USER]

Example:

$ dqr-ping
OK
INFO:dqr-xfer:Login: 1.33, Ping: 0.30, status:200
OK
INFO:dqr-xfer:Login: 1.33, Ping: 0.27, status:200
OK
INFO:dqr-xfer:Login: 1.33, Ping: 0.27, status:200
INFO:dqr-xfer:Success = 100.0%
INFO:dqr-xfer:Elapsed time: 2.7s

dqr-upload

Uploads files or directories for a single DQR event to the central site. Supports both SCP (default) and HTTPS transfer.

usage: dqr-upload [-h] [-v] [-V] [-q] [-s SERVER] [--port PORT]
                  [--retries RETRIES] [--retry-wait RETRY_WAIT]
                  [-g GRACEID] [-u UPLOADER] [-p PROJECT] [--user USER]
                  [--revision REVISION] [--subdir SUBDIR] [--config CONFIG]
                  [--set-dir SET_DIR] [--use-https] [--upload-path UPLOAD_PATH]
                  [--remote-command REMOTE_COMMAND] [--temp-dir TEMP_DIR]
                  [files ...]

Key arguments:

dqr-batch-upload

Scans a DQR result directory and calls dqr-upload for each event found within it. Useful for bulk transfers.

usage: dqr-batch-upload [-h] [-v] [-V] [-q] [-d INDIR] [-p PROJECT]
                        [-u UPLOADER] [--set-dir] [-s SERVER] [--test]

Authentication

dqrxfer uses ciecplib for Shibboleth/ECP authentication over HTTPS. For SCP transfers, standard Kerberos credentials are used.

Obtain a Kerberos ticket before running any command:

kinit albert.einstein@LIGO.ORG

If no Kerberos ticket is available, you will see:

klist: No credentials cache found

License

This project is licensed under the GNU General Public License v3.0.