dq-genesis

Pipeline construction and execution for the creation of the data quality vector.

dq-genesis parses a YAML DQ flag config, assembles an sgn.Pipeline from the evaluators and transforms provided by sgn-dq, and runs it against frame data (offline frame cache), shared-memory frames (online), or an Arrakis stream. The output is a uint32 DQG-DQ_VECTOR channel written to a GWF frame stream.

Install

pip install -e /path/to/dq-genesis

dq-genesis depends on sgn-dq (evaluators, Not, Parity), sgn-ts, sgn-ligo, and sgn.

Quick start

# Validate a config
dq-genesis --flags configs/dqmodule_h1.yaml --dry-run

# Offline run against a frame cache
dq-genesis --flags configs/dqmodule_h1.yaml \
  --gps-start 1234567890 --gps-end 1234568000 \
  --frame-cache h1.cache --output-dir /tmp/out

# Bounded replay from Arrakis (omit --gps-start/--gps-end for a live stream)
dq-genesis --flags configs/dqmodule_h1.yaml \
  --arrakis --gps-start 1234567890 --gps-end 1234568000 \
  --output-dir /tmp/out

See src/dqgenesis/cli.py --help for the full argument surface.

Output destinations

The assembled DQ vector can be fanned out to one or more sinks. Select any combination of the --publish-* flags; if none is given it defaults to disk, preserving the original frame-writing behaviour.

• --publish-disk — write the DQ vector to GWF frame files. Control the destination with --output-dir (and --write-path, --frame-duration).
• --publish-arrakis — publish the DQ vector channel back to an Arrakis server. Requires --publisher-id; pass --replay-id to publish into a replay namespace.

# Save to GWF frames (default)
dq-genesis --flags configs/dqmodule_h1.yaml \
  --gps-start 1234567890 --gps-end 1234568000 \
  --frame-cache h1.cache --publish-disk --output-dir /tmp/out

# Publish back into Arrakis
dq-genesis --flags configs/dqmodule_h1.yaml \
  --arrakis --gps-start 1234567890 --gps-end 1234568000 \
  --publish-arrakis --publisher-id <publisher_id>

# Both at once — the DQ vector is fanned out to disk and Arrakis
dq-genesis --flags configs/dqmodule_h1.yaml \
  --arrakis --gps-start 1234567890 --gps-end 1234568000 \
  --publish-disk --output-dir /tmp/out \
  --publish-arrakis --publisher-id <publisher_id>

The output channel is named <IFO>:DQG-DQ_VECTOR by default; override it with --output-channel.