pe-automator 0.21.0

PE Automator is a Python package for automating the setup and execution of parameter estimation (PE) runs using Bilby Pipe.

PE Automator

A plugin-based workflow tool for gravitational-wave parameter estimation using bilby-pipe. Three run modes are supported: real event data, Gaussian-noise injections, and real-noise injections from GWOSC.

---

Installation

conda create -n pe_automator python=3.10 setuptools_scm
conda activate pe_automator
make install

---

Project directory layout

Real-data runs

data_path/
├── configs/
│   └── {eventname}_config.ini      # bilby-pipe config with event metadata
├── framefiles/
│   ├── {eventname}_H1.gwf          # pre-downloaded strain frames (git-lfs)
│   ├── {eventname}_L1.gwf
│   └── {eventname}_V1.gwf
├── psds/
│   └── {eventname}_{det}_psd.txt
├── spline_cal_envs/
├── templates/
│   └── bilby_config.tpl.ini        # Jinja2 template shared by all modes
└── project/
    ├── project.json                 # gitlab_url, gitlab_project
    └── allocations.json             # cluster allocation profiles

The framefiles are stored as git-lfs objects. Install and initialise git-lfs before cloning:

git lfs install

Injection runs (gaussian_noise / real_noise)

data_path/
├── inj_config/
│   └── {inj_set}/
│       ├── injection_params.json   # list of N injection parameter dicts
│       ├── injection_points.json   # (real_noise only) list of M GPS points
│       └── sample_config.ini       # bilby-pipe base config for this inj set
├── psds/
│   └── {inj_set}_{det}_psd.txt    # per-detector PSD (2-column: freq, PSD)
├── templates/
│   └── bilby_config.tpl.ini
└── project/
    ├── project.json
    └── allocations.json

GWF frame files for injections are generated at run time and written directly into runs/{label}/framefiles/; data_path is never modified.

---

CLI reference

setup — create and submit PE runs

pe_automator setup <name> --mode <mode> [options]

<name> is the event name (real_data) or injection set name (gaussian_noise / real_noise).

Common options

Flag	Default	Description
--mode	real_data	real_data, gaussian_noise, or real_noise
--data_path	./data	Project data directory
--run_label	(required)	Label appended to the run directory name
--approximant	(required)	LALSim waveform approximant
--allocation	(required)	Allocation key from allocations.json
--user	(required)	SSH username on the cluster
--conda_env	(required)	Conda environment on the cluster
--private_token	(required)	GitLab personal access token (api scope)
--account		SLURM account
--partition		SLURM partition
--qos		SLURM QoS
--memory	300	Memory per node (GB)
--walltime	71:40:00	SLURM wall-clock limit
--cpu	allocation default	CPUs per node
--npoint	1000	Dynesty npoints
--nact	50	Dynesty nact
--naccept	60	Dynesty naccept
--maxmcmc	20000	Dynesty maxmcmc
--dry_run	False	Generate files locally; skip upload and submission
--distance_marginalization / --no-distance_marginalization		Enable/disable distance marginalisation
--priors		Path to custom priors file
--mode_array		Waveform mode array override
--wf_min_f		Waveform minimum frequency (Hz)
--wf_ref_f		Waveform reference frequency (Hz)
--min_f		Analysis minimum frequency (Hz)
--comment		Free-text comment added to the GitLab issue

Injection-only options (gaussian_noise and real_noise)

Flag	Default	Description
--flow	20.0	Low-frequency cutoff for noise/waveform generation (Hz)
--f_ref	20.0	Reference frequency for waveform generation (Hz)
--force_regenerate	False	Regenerate GWF files even if they already exist

real_noise only

Flag	Default	Description
--fetch_buffer	16	Extra seconds fetched around the injection window

GitLab token: select the api scope. See GitLab docs.

---

Quick start examples

1. Real-data run

pe_automator setup GW150914 \
    --mode real_data \
    --data_path ./data \
    --run_label run1 \
    --approximant IMRPhenomXPNR \
    --account uib107 --partition gpp --qos gp_resa \
    --user resh000428 --conda_env pe_env \
    --private_token <token> \
    --allocation AECT-2025-2-0029

2. Gaussian-noise injection runs

Generates N × M jobs where N = entries in injection_params.json and M = noise_seeds listed in each entry.

pe_automator setup inj_set1 \
    --mode gaussian_noise \
    --data_path ./data \
    --run_label run1 \
    --approximant IMRPhenomXPNR \
    --flow 20.0 --f_ref 20.0 \
    --account uib107 --partition gpp --qos gp_resa \
    --user resh000428 --conda_env pe_env \
    --private_token <token> \
    --allocation AECT-2025-2-0029

Add --dry_run to generate configs and GWF files locally without uploading.

Required files under data_path:

Path	Description
inj_config/{inj_set}/injection_params.json	N injection parameter dicts (each with a noise_seeds list)
inj_config/{inj_set}/sample_config.ini	Base bilby-pipe config (detectors, priors, …)
psds/{inj_set}_{det}_psd.txt	PSD for each detector

3. Real-noise injection runs (GWOSC)

Generates N × M jobs where N = entries in injection_params.json and M = entries in injection_points.json.

pe_automator setup inj_set1 \
    --mode real_noise \
    --data_path ./data \
    --run_label run1 \
    --approximant IMRPhenomXPNR \
    --flow 20.0 --f_ref 20.0 --fetch_buffer 16 \
    --account uib107 --partition gpp --qos gp_resa \
    --user resh000428 --conda_env pe_env \
    --private_token <token> \
    --allocation AECT-2025-2-0029

Additional required file:

Path	Description
inj_config/{inj_set}/injection_points.json	M GPS injection points with per-detector time shifts

---

Injection parameter format (injection_params.json)

[
  {
    "mass_1": 35.6,
    "mass_2": 30.4,
    "luminosity_distance": 450.0,
    "geocent_time": 1187008882.43,
    "ra": 1.375,
    "dec": -1.211,
    "psi": 0.0,
    "theta_jn": 0.4,
    "chi_1": 0.0,
    "chi_2": 0.0,
    "noise_seeds": [1000, 2000, 3000]
  }
]

noise_seeds (Gaussian-noise mode) controls how many noise realisations are generated per injection. Spin can be given as aligned (chi_1/chi_2), bilby spherical (a_1, tilt_1, …), or Cartesian (spin_1x/y/z).

Injection point format (injection_points.json) — real_noise only

[
  {
    "gps_time": 1187008882.43,
    "shifts": {"H1": 0.0, "L1": 3.14, "V1": 7.0},
    "channel": {
      "H1": "H1:GWOSC-4KHZ_R1_STRAIN",
      "L1": "L1:GWOSC-4KHZ_R1_STRAIN",
      "V1": "V1:GWOSC-4KHZ_R1_STRAIN"
    }
  }
]

Non-zero shifts fetch each detector's background from a different GPS time, breaking coherence for background estimation while the injected signal stays coherent at gps_time.

---

Bundled PSDs

from pe_automator.injection_generator.psds import get_psd_path
psd = get_psd_path("AplusDesign_O5")

Name	Detector
AplusDesign_O5	A+ (LIGO O5 design)
AdV_DESIGN	Advanced Virgo design
aLIGO_ZERO_DET_high_P	aLIGO zero-det high power
ET_D	Einstein Telescope D

---

Other CLI commands

monitor — track job status

pe_automator monitor \
    --private_token <token> \
    --ssh_key ~/.ssh/id_rsa \
    --data_path ./data

rescue — resubmit a failed job

pe_automator rescue <issue_number> \
    --data_path ./data \
    --private_token <token> \
    --walltime 47:00:00

setup_env — deploy a conda environment on the cluster

pe_automator setup_env pe-0.0.1beta1 \
    --source_env my_env.tar.gz \
    --source_remote [e-mail] \
    --data_dir ./data

post_process — post-process PE results

pe_automator post_process \
    --results_dir ./results \
    --data_dir ./data \
    --output_dir .

dlogz — check sampler convergence

pe_automator dlogz \
    --private_token <token> \
    --ssh_key ~/.ssh/id_rsa \
    --data_path ./data