STEREOGRAPH

Stereoscopic event reconstruction using graph neural networks applied to CTAO

This project applies Graph Neural Networks (GNNs) to the stereoscopic event reconstruction for the Cherenkov Telescope Array Observatory (CTAO). We use GNNs to combine data from multiple telescopes. This project provides a framework to easily apply models, train, perform inference, and plot IRFs from a configuration file.

# Usage

Stereograph's CI/CD pipelines generate containers that provide a ready-to-use environment for development and production. These containers are uploaded to Stereograph's container registry. A set of containers is available for each branch in the repository, and the containers are re-build for any commit that involves modifications made to the environment or dockerfiles files (anything in the docker directory). The containers include pytorch and cuda dependencies to do computation on nvidia gpus, in addition to a big stack of scientific computing software. They typically require several GBs of storage.

For Users

The use of apptainer is recommended. To obtain the production image use the following command:

apptainer pull docker gitlab-registry.in2p3.fr/gammalearn/stereograph/stereograph:main

This command will create a .sif file, which provides a ready-to-use Stereograph environment Note : apptainer can use several GB of disk storage as cache when building the .sif file. The default cache is usually located in ~/.apptainer/cache. The location can be changed by setting the APPTAINER_CACHEDIR environment variable . The apptainer cache can be cleaned using: \

apptainer cache clean  

Run an Experiment

To run an experiment, you need to set your configuration file. See the example here:

• Configuration File

The configuration file is a YAML file that defines all the necessary parameters to train the model using Stereograph.

You can specify:

• The network parameters
• The input (node features) you want to use
• The number of telescopes (from 1 to 11, plus 19 and 35)
• The number of events (since the dataset is large, you can select a specific number of events, or set the variable to null to use the entire dataset if you have sufficient resources)

Note: The configuration file used to produce the graphs must be the same as the one used to train the model, especially with regard to the data.

You can now run your experiment using apptainer run ince apptainer containers are read-only by default, you will need to mount the paths to your input and output files. To use nvidia gpus, you will need to specify the --nv option as well. A typical command example:

# Run the experiment in the container
# Parameters:
# --nv                  to use nvidia gpus from inside the container
# CUDA_VISIBLE_DEVICES  env variable used by pytorch to discover the gpus
# NUMBA_CACHE_DIR       a writable directory where numba can store its compiled functions
#                       (needs to be outside of the container, which is read-only)
# CTAPIPE_CACHE         ctapipe needs a writable place, to store its downloaded files.
# Mounts: input (data and settings file) and output directories
# 
# We call the stereograph entrypoint directly in /opt/conda/bin, because micromamba is not initialized
# inside the container for a new user (and every user is new, since with apptainer the user remains the same
# as the user on the host system by default (only users defined in the containers are known))
apptainer run \
    --nv \
    --env "CUDA_VISIBLE_DEVICES=$CUDA_VISIBLE_DEVICES" \
    --env "NUMBA_CACHE_DIR=/tmp/NUMBA" \
    --env "CTAPIPE_CACHE=/tmp/CTAPIPE" \
    --mount type=bind,source=/path/to/input/data_dir/,destination=/corresponding/path/in/container/ \
     $CONTAINER \
     bash -c "\
    $PYTHON	$TRAIN_FILE --config_file $CONFIG_FILE_ENERGY  --gamma_file gamma.graphs --overwrite && \  #example of a command to run a training