HARD-lib 1.0.5

Pre-requisites

1. Create an ECMWF account to access the ERA5 CDS API (for setup go to cds.climate.copernicus.eu). This is needed for wind-field information.

2. Create an Alaskan Satellite Fascility (ASF) account so that you may querry their API using asf_search. Store your ASK credentials in a .netrc file. This will be needed to download data from ASF.

An example .netrc:

machine asf
login YOURUSERNAME
password YOURPASSWORD

Installation

Create a new environment and activate

conda create -n ENV_NAME python==3.12

conda activate ENV_NAME

Conda install GDAL (not enabled from pip)

conda install GDAL

For development

Clone HARD repository

git clone https://gitlab.tudelft.nl/yyuan6/hard.git

Navigate to cloned folder and install an editibale version

cd HARD

pip install -e .

For easy usage

Download and install the (likely outdated) package from pip

pip install HARD-lib

Usage

Python

For Python tutorials refer to the notebook tutorials in hard/Tutorial/

Command Line Interface (CLI)

To run an automated download, processing and saving exercise using a bash script go to either:

• hard/HARD/CLI/download_process_save_local_example.sh
• hard/HARD/CLI/download_process_save_DelftBlue_example.sh

Please copy the files before editing and do not track them in this git repository.

NOTE

When submitting the bash script on Delft Blue, please make sure a logs subdirectory exists in the same location as the .sh file, e.g.:

mkdir -p logs
sbatch /home/.../hard/HARD/CLI/download_process_save_DelftBlue_example.sh

NRCS Re-calibration

Sentinel-1 WV

The Sentinel-1 WaVe mode calibration experienced significant changes throughout the mission lifetime, e.g. check figure 6/7 on page 72 in Sentinel-1 annual performance report 2024. These calibration changes are NOT retroactively applied, such that new and old scenes may experience radiometric offsets purely due to the used calibration scheme.

To Counteract this, a re-calibration needs to be performed. Here we set out to perfom a re-calibration using the most up-to-date Sentinel-1 calibration and processing files. The re-calition is an optional step, making using of xsar's existing template. Currently xsar can already re-calibrate certain Sentinel-1 modes, but not yet WaVe mode (and IW mode anecdotally yields peculiar results). Thus, updates to xsar may depreciate HARD's recalibration.

NOTE: The re-calibration is relative, and new calibration files may become available for past scenes, necessitating yet another re-calibration. Nonetheless, the follow steps should take care of the most blatant calibration-induced artefacts which primarily occur prior to 12-05-2020.

NOTE: WV-mode NRCS re-calibration is only useful between 13-03-2018 - 12-05-2020. Prior to 13-03-2018 no valid radiometric normalization was included in the Sentinel-1 processor, which, as you can see here sar-mpc.eu/processor/ipf was in version 2.82 for TOPS mode, but only fully from 2.90 for WV/SM modes. Beyond 12-05-2020 there is negligeable change after re-calibration as the system introduced then is still being used now (May 2026). For new updates check sar-mpc.eu.

Step 1.

Download the individual calibration and processing files from Sentinel-1's Mission Performance Center sar-mpc.eu. This can be done automatically by running

python src/HARD/CLI/download_S1_calibration.py --path_storage /location/containing/the/recalibration/files/

Step 2.

Configure xsar's config file, which should be located at ~/.xsar/config.yml, to direct to necessary files and directory such as:

data_dir: /tmp
auxiliary_dir: /project/fluxsar/Data/Sentinel-1/calibration/Auxiliary/
path_dataframe_aux: /project/fluxsar/Data/Sentinel-1/calibration/Auxiliary/active_aux.csv

Where auxiliary_dir refers to the parent directory where the following subfolders live: /S1A_AUX_CAL/, /S1A_AUX_PP1/,/S1B_AUX_CAL/, /S1B_AUX_PP1/, /S1C_AUX_CAL/, /S1C_AUX_PP1/, /S1D_AUX_CAL/, /S1D_AUX_PP1/,

Part 3.

When running ProcessWV inside the download_preprocess_save_WV.py file, make sure to supply

ProcessWV(
    ...
    custom_recalibration=True,
    ...
)

Re-calibrated processed imagettes will now contain new attributes aux_cal_new and aux_pp1_new in addition to the already exisiting aux_cal and aux_pp1, to clarify which fiels were used to perform the re-calibration (and, perhaps, whetehr re-calirbation should be performed again if the "new" files are now themselves outdated)