SGN-LLAI

SGN pipelines for gravitational wave event processing with GraceDB integration.

Quick Start

Prerequisites: sgn-sdk running with Kafka, GraceDB, InfluxDB, and Grafana.

# 1. Install
pip install -e .

# 2. Create Kafka topics
make topics-reset

# 3. Set up Grafana dashboards
make grafana-setup

# 4. Run pipeline (Terminal 1)
make run

# 5. Publish mock events (Terminal 2)
make publish

# 6. Run Bayestar (Terminal 3)
make run-bayestar

Verify:

• GraceDB: http://127.0.0.1:8080 (events G######, superevents S######)
• Grafana: http://localhost:3000 (login: admin/sgnl)

For all available commands: make help

Pipelines

Superevent Pipeline (make run)

Clusters GW events by GPS time and creates superevents in GraceDB:

MockGWEventSource → Kafka → SuperEventCreator → Kafka (superevents)
                                   │           → Kafka (skipped)
                                   ↓
                              GraceDB API
                                   │
                                   ↓
                          ScaldMetricsSink → InfluxDB → Grafana

Bayestar Pipeline (make run-bayestar)

Generates sky localization maps for superevents:

Kafka (superevents) → BayestarProcessor → GraceDB (skymap upload)
                             │
                             ↓
                     ScaldMetricsSink → InfluxDB → Grafana

Run both pipelines together:

# Terminal 1
make run

# Terminal 2
make run-bayestar

# Terminal 3
make publish

Configuration

Override defaults with make arguments:

make run KAFKA_SERVER=localhost:9092
make publish PUBLISH_RATE=10.0

Bayestar-specific:

Kafka Topics

make topics        # create
make topics-reset  # delete and recreate
make topics-list   # list all

Monitor topics:

make listen-events   # input
make listen-supers   # superevents
make listen-skipped  # skipped

Output Formats

Superevents (gw-superevent-topic)

On creation:

{
  "action": "created",
  "superevent_id": "S241009abc",
  "preferred_event": "G123456",
  "t_0": 1412546713.52,
  "gw_events": ["G123456"],
  "g_event": {"graceid": "G123456", "snr": 12.5, "pipeline": "gstlal"}
}

On update (higher SNR arrived):

{
  "action": "updated",
  "superevent_id": "S241009abc",
  "preferred_event": "G123457",
  "previous_preferred": "G123456",
  "gw_events": ["G123456", "G123457"],
  "g_event": {"graceid": "G123457", "snr": 15.0, "pipeline": "pycbc"}
}

Skipped Events (gw-skipped-topic)

{
  "gpstime": 1412546713.60,
  "pipeline": "spiir",
  "snr": 10.0,
  "reason": "lower_snr",
  "superevent_id": "S241009abc",
  "preferred_snr": 12.5
}

Development

make test       # run tests
make lint       # run linter
make format     # format code
make all        # all checks

Troubleshooting

# Reset Kafka topics
make topics-reset

# Check sgn-sdk services
cd ../sgn-sdk && docker compose ps

# Restart if needed
docker compose down && docker compose up -d