sgn-manifold 0.1.0.post0

Geometric tools for gravitational wave data analysis

THIS PACKAGE IS DEPRECATED FUTURE DEVELOPMENT SHOULD BE IN https://git.ligo.org/greg/sgn-manifold

Manifold

Tutorial: Generating a CBC Template Bank

This tutorial walks through the complete process of generating a compact binary coalescence (CBC) template bank using Manifold, from initial seed bank generation through parallel refinement and final testing.

Prerequisites

You'll need the O4_projected_psds.xml.gz file from the examples directory. Copy it to your working directory:

cp examples/O4_projected_psds.xml.gz .

Step 1: Create a Configuration File

Create a file called bank.yaml with the following contents. This configuration uses a smaller parameter space for faster execution:

# Mass parameters (in solar masses)
m1: [10, 100]        # Primary mass range
m2: [10, 100]        # Secondary mass range
M: [20, 200]         # Total mass range
mc: [8, 200]         # Chirp mass range
q: [1.0, 10.0]       # Mass ratio range (m1/m2)

# Spin parameters
chi: [-0.5, 0.5]     # Effective aligned spin parameter
ns-mass: [0, 3.0]    # Neutron star mass threshold
ns-s1z: [-0.05, 0.05]  # Neutron star spin limit

# Frequency parameters
freq: [10, 512]      # Frequency range [flow, fhigh] in Hz
max-duration: 128    # Maximum signal duration in seconds

# PSD and instrument
psd-xml: O4_projected_psds.xml.gz
instrument: L1       # Options: L1, H1, V1

# Bank generation parameters
mm: 0.03             # Maximum mismatch (3% mismatch = 97% minimum match)
reuse-g-mm: 0.10     # Mismatch for reusing metric calculations
min-coord-vol: 1000  # Minimum coordinate volume for rectangles
min-depth: 7         # Minimum tree depth for placement
approximant: IMRPhenomD

# Processing options
trim: true           # Trim templates outside constraints
verbose: true

Step 2: Generate the Initial Seed Bank

Generate a coarse seed bank with ~1000 templates:

manifold-cbc-bank --yaml bank.yaml --output-h5 SEED.h5 --max-num-templates 1000

You should see output showing the placement of templates, ending with something like:

900 <Rectangle [(...), (...), (...)]> 1.0
1000 <Rectangle [(...), (...), (...)]> 1.0
Trimming templates outside constraints...
1024 templates before
987 templates after
Saving bank to SEED.h5...
Bank saved successfully with 987 templates

Step 3: Divide the Seed Bank into Groups

Split the seed bank into groups for parallel refinement (e.g., 10 groups):

manifold-cbc-bank-group --bank SEED.h5 --num-groups 10

This creates:

• H1L1V1-00000_MANIFOLD_SEED-0-2000000000.h5 through H1L1V1-00009_MANIFOLD_SEED-0-2000000000.h5 (seed groups)
• H1L1V1-MANIFOLD_SEED_MANIFEST-0-2000000000.json (manifest file)

Step 4: Refine Each Seed Group in Parallel

For each seed group, generate a refined bank with much smaller rectangles. This step can be parallelized across multiple cores/machines:

# Process all 10 groups (can be run in parallel)
for i in $(seq -f "%05g" 0 9); do
    manifold-cbc-bank --yaml bank.yaml \
        --seedbank H1L1V1-${i}_MANIFOLD_SEED-0-2000000000.h5 \
        --output-h5 H1L1V1-${i}_MANIFOLD-0-2000000000.h5 \
        --max-num-templates 1 \
        --min-coord-vol 0.0001
done

Each refined bank will have many more templates (~5000-6000 each) covering its portion of the parameter space more finely.

Step 5: Combine the Refined Banks

Merge all refined banks into a single bank:

manifold-cbc-bank-add --output-h5 H1L1V1-MANIFOLD_ADD-0-2000000000.h5 \
    H1L1V1-00000_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00001_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00002_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00003_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00004_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00005_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00006_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00007_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00008_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00009_MANIFOLD-0-2000000000.h5

Step 6: Check and Improve Coverage (Parallel)

Check the bank's coverage and split poorly covered templates. This can also be parallelized by processing intervals:

# Process 10 intervals in parallel (each checks 1/10th of the bank)
for i in $(seq 0 9); do
    manifold-cbc-bank-check-coverage \
        --bank H1L1V1-MANIFOLD_ADD-0-2000000000.h5 \
        --interval-index $i \
        --num-intervals 10 \
        --neighbors 10000 \
        --target-match 0.965 \
        --output-bank H1L1V1-MANIFOLD_COVERAGE_$(printf "%02d" $i)-0-2000000000.h5 \
        --verbose
done

Key parameters:

• --interval-index: Which interval to process (0-based)
• --num-intervals: Total number of intervals
• --neighbors: Number of neighbors to use as buffer (should be large, e.g., 10000)
• --target-match: Minimum match threshold (0.965 = 96.5% match)

Each interval will check its portion of the bank and split templates that don't meet the target match.

Step 7: Combine Coverage-Improved Banks

Merge all coverage-improved intervals:

manifold-cbc-bank-add --output-h5 H1L1V1-MANIFOLD-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_00-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_01-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_02-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_03-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_04-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_05-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_06-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_07-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_08-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_09-0-2000000000.h5

This is your final template bank!

Step 8: Test the Bank

Generate test injections to verify bank performance. Again, this can be parallelized:

# Generate 10 test sets in parallel (100 injections each)
for i in $(seq -f "%02g" 1 10); do
    manifold-cbc-bank-test \
        --bank H1L1V1-MANIFOLD-0-2000000000.h5 \
        --output-h5 ${i}_test.h5 \
        --num-injections 100 \
        --verbose
done

Step 9: Combine and Plot Test Results

Combine all test results:

manifold-cbc-bank-test-add --output-h5 test.h5 \
    01_test.h5 02_test.h5 03_test.h5 04_test.h5 05_test.h5 \
    06_test.h5 07_test.h5 08_test.h5 09_test.h5 10_test.h5

Generate diagnostic plots:

manifold-cbc-bank-test-plot --test test.h5

This creates several plots showing:

• Match vs injection parameters (m1, m2, chirp mass, mass ratio, etc.)
• Mismatch histogram
• 2D match distributions

Step 10 (Optional): Add Constraint Sets and Template IDs

If you need to prepare the bank for online analysis, add constraint sets and assign template IDs:

manifold-cbc-bank-constrain \
    --bank H1L1V1-MANIFOLD-0-2000000000.h5 \
    --output-h5 H1L1V1-MANIFOLD_CONSTRAINED-0-2000000000.h5 \
    --yaml constrain.yaml

where constrain.yaml contains:

m1: [12, 98]
m2: [12, 98]
mc: [10, 90]
q: [1.0, 8.0]
M: [24, 196]
chi: [-0.4, 0.4]
ns-mass: [0, 3.0]
ns-s1z: [-0.04, 0.04]
assign-ids: true

Step 11 (Optional): Generate Mass Model

Create a mass model for the bank:

manifold-cbc-bank-mass-model \
    --bank H1L1V1-MANIFOLD_CONSTRAINED-0-2000000000.h5 \
    --output-h5 H1L1V1-MANIFOLD_MASS_MODEL-0-2000000000.h5 \
    --yaml mass_model.yaml

where mass_model.yaml contains:

mass_model:
    model: salpeter
    m_min: 0.8

Step 12 (Optional): Convert to XML

Convert the bank to XML format for use with other tools:

manifold-cbc-bank-to-xml \
    H1L1V1-MANIFOLD_CONSTRAINED-0-2000000000.h5 \
    --output-xml H1L1V1-MANIFOLD-0-2000000000.xml.gz

Using Make for Automation

You can automate the entire workflow using a Makefile. Create a file called Makefile:

all: test

# Step 2: Generate seed bank
SEED.h5: bank.yaml O4_projected_psds.xml.gz
	manifold-cbc-bank --yaml bank.yaml --output-h5 $@ --max-num-templates 1000 --min-coord-vol 1000

# Step 3: Divide into groups
H1L1V1-MANIFOLD_SEED_MANIFEST-0-2000000000.json: SEED.h5
	manifold-cbc-bank-group --bank SEED.h5 --num-groups 10

# Step 4: Refine each group (pattern rule)
H1L1V1-%_MANIFOLD-0-2000000000.h5: H1L1V1-MANIFOLD_SEED_MANIFEST-0-2000000000.json
	manifold-cbc-bank --yaml bank.yaml --output-h5 $@ \
		--seedbank H1L1V1-$*_MANIFOLD_SEED-0-2000000000.h5 \
		--max-num-templates 1 --min-coord-vol 0.0001

# Step 5: Combine refined banks
H1L1V1-MANIFOLD_ADD-0-2000000000.h5: \
		H1L1V1-00000_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00001_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00002_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00003_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00004_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00005_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00006_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00007_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00008_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00009_MANIFOLD-0-2000000000.h5
	manifold-cbc-bank-add --output-h5 $@ $^

# Step 6: Check coverage for each interval (pattern rule)
H1L1V1-MANIFOLD_COVERAGE_%-0-2000000000.h5: H1L1V1-MANIFOLD_ADD-0-2000000000.h5
	manifold-cbc-bank-check-coverage \
		--bank $< \
		--interval-index $* \
		--num-intervals 10 \
		--neighbors 10000 \
		--target-match 0.965 \
		--output-bank $@ \
		--verbose

# Step 7: Combine coverage-improved banks
H1L1V1-MANIFOLD-0-2000000000.h5: \
		H1L1V1-MANIFOLD_COVERAGE_00-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_01-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_02-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_03-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_04-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_05-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_06-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_07-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_08-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_09-0-2000000000.h5
	manifold-cbc-bank-add --output-h5 $@ $^

# Step 8: Generate test injections (pattern rule)
%_test.h5: H1L1V1-MANIFOLD-0-2000000000.h5
	manifold-cbc-bank-test --bank $< --output-h5 $@ --num-injections 100 --verbose

# Step 9: Combine test results
test.h5: 01_test.h5 02_test.h5 03_test.h5 04_test.h5 05_test.h5 \
         06_test.h5 07_test.h5 08_test.h5 09_test.h5 10_test.h5
	manifold-cbc-bank-test-add --output-h5 $@ $^

# Step 9: Plot test results
test: test.h5
	manifold-cbc-bank-test-plot --test $<

clean:
	rm -rf *.h5 *.png *.json

.PHONY: all test clean

Then simply run:

make -j 10  # Run with up to 10 parallel jobs

Performance Notes

Parallel Processing:

• Steps 4, 6, and 8 can be parallelized across multiple cores or machines
• For Step 6 (coverage checking), use a large --neighbors value (e.g., 10000) to ensure accurate results
• Small differences in template counts between single and parallel runs are expected and acceptable

Parameter Space Size:

• Smaller mass ranges and spin ranges = faster execution
• The tutorial configuration should complete in ~30-60 minutes on a modern machine
• For production banks, use larger ranges (e.g., m1/m2: [3, 200], chi: [-0.95, 0.95])

Memory Usage:

• Each refined bank group uses ~1-2 GB of memory
• Coverage checking uses ~2-4 GB per interval
• Testing uses ~500 MB per test set