lst-dl0-converter

lst-dl0-converter is a tool designed to convert LST-1 R0 data files into CTAO DL0 format. This makes it possible to analyze LST-1 data using standard CTAO software (DPPS).

Most LST commissioning data has been written as uncalibrated R0 data using an older definition of the R1 event data structure (ProtoR1).

To convert this data to DL0 compliant with the data model and the current ACADA to DPPS ICD, several steps are necessary:

• Calibrate the waveforms applying DRS4 corrections and calibration from adc counts to photo electrons
• Computing pixel_timeshift from DRS4 time correction and flatfielding
• For older data: compute best event timestamp from available sources.
• For older data: determine R1 event_type from lower level trigger bit information
• Optional: using lstchain muon analysis results to tag muon candidate events
• Optional: using lstchain DVR pixel selection to apply DVR. Most data still available in the onsite cluster is already data volume reduced, but not calibrated (R0V), so this step should not be necessary in most cases.
• Write out the calibrated, tagged events to DL0 files using the directory structure and filenames as prescribed by the ACADA-DPPS ICD.

Reading the R0 data and calibrating it to R1 is handled by ctapipe_io_lst, converting the R1 to DL0 is performed by code in this repository and most of the writing is handled by the acada-mock package.

Applying the DRS4 corrections is compute intensive. For this reasons, the converter will work best when converting LST subruns in parallel to equivalent chunks. This however violates the chunking prescribed by the ACADA-DPPS ICD:

• Files with shower candidates will not follow the event_id based chunking and will contain roughly 13000 events, that's the LST chunk size of 13250 minus the calibration events of that chunk.
• Files with calibration events will be much smaller than the chunk size, as any given subrun will only contain hundreds of calibration events.
• Files with the muon event candidates will be very small.

Since all LST data that needs this conversion is monoscopic LST-1 data, this does not seem to be an issue. The chunking could be made to follow the ICD in a second step, but this is currently not planned.

Finding calibration files

The converter needs several files with calibration coefficients. While the converter itself does not depend on lstchain and it is at least currently not possible to have lstchain in the same environment due to different requirements on ctapipe, this repository provides a script using lstchain to find the needed calibration files in the IT cluster and put them into a config file for the converter.

Use a separate environment with lstchain and run the script to find the files for a given run:

$ python scripts/create_config_for_run.py 11961 > config_11961.yaml