candies 0.1.1

Contents

• Rationale
• Features
• Installation
• Quick Guide

Rationale

The SPOTLIGHT project is a multibeam, commensal survey for FRBs and pulsars soon to be undertaken at the GMRT. It is estimated that it will produce almost 1 PB worth of data per day¹. Therefore, it is obviously impossible for a mere human being, or even a group of them, to sift through and classify all the millions and billions of resulting FRB candidates. Thus, we plan to use FETCH, a ML/DL-based classifier. However, every classifier requires some features per candidate, for both training and classification. FETCH requires two features to classify candidates: their dedispersed dynamic spectrum, and their DM transform (DMT). The authors of FETCH recommend using utility scripts packaged with their associated library, your, to generate them: your_candmaker.py and your_h5plotter.py. However, these scripts were developed and tested using data taken from high-frequency telescopes, such as the VLA. Thus, for low-frequency data obtained via the GMRT, the features generated by these scripts proved to be sub-optimal (see the plot below). The scripts were also much slower than real time despite using the GPU.

Fig: Variation of the DM transform (DMT) over different frequency bands at the GMRT. From left to right, we are going from lower to higher frequency. For real signals, we expect a bow-tie pattern in the DMT, which becomes more and more difficult to observe at lower frequencies. Plots in the first row were created using your's your_candmaker.py, while plots in the second row were created via candies for the same exact bursts.

Thus, we decided to develop candies, our feature extraction library for FRB candidates. Primarily developed by me as a part of my (ongoing) PhD (and with the help of engineers from NVIDIA), it hopes to be a better and faster variant of your's scripts used for the same thing.

Features

candies plans to introduce the following features:

• Zooming into the DMT (see plots above).
• Dealing with band-limited bursts.
• Support for GMRT's shared memory based ring buffers.

Currently, we only support processing data stored in the SIGPROC filterbank format, but support for additional formats might be added if there is any interest from the community at large.

Installation

candies is on PyPI, and hence can be installed by simply running:

pip install candies

In case you are interested in the cutting edge, you can also git clone and install it directly via Github:

git clone https://github.com/astrogewgaw/candies
cd candies
pip install -e .

where the -e flag is for an editable install; that is, any changes to the source code will reflect directly in your installation. If you don't want that, just remove the flag, or you can go ahead and use the more direct method:

pip install git+https://github.com/astrogewgaw/candies

Note: To get plotting working in candies, you will have to install proplot. It is recommended that you do not do so via PyPI (that is, do not do pip install proplot). Instead, use the following command to install proplot (for matplotlib v3.6 and above):

pip install git+https://github.com/proplot-dev/proplot.git@refs/pull/459/head

or (for matplotlib v3.5 and below):

pip install git+https://github.com/proplot-dev/proplot

This is due to several issues with the main branch of proplot (see #461, #451, #453, #451, #432, #418, and #309), which is incompatible with the latest versions of matplotlib and other packages. Installing from the as-of-yet-unmerged pull request above solves this issue. For matplotlib versions below v3.5, the main branch works fine. In case proplot is not installed, plotting will fail, and raise an error.

Quick Guide

For help with how to use candies, just type and run candies or candies --help. To process a list of FRB candidates stored as a CSV file, just use the command:

candies make /path/to/candidate/list

To see what additional options you can use, just type candies make --help. To plot one or several HDF5 files generated as output, just run:

candies plot /path/to/HDF5/files

Once more, type candies plot --help to see what options are available to you. Note that the plotting command will only work if you have proplot installed; see the note above regarding the same. You can also generate a list of candidates on the terminal, via:

candies list /path/to/HDF5/files

This command is useful is you just have a bunch of HDF5 files generated via candies, and you want to have a quick look at their details. You can also save these details as a CSV file, for later use.