Read the ABSCAL.HIS file and visualize the results of the absolute calibration with liquid nitrogen of an RPG microwave radiometer

Introduction

📖 Documentation: https://eval-ac.readthedocs.io (user guide, Python API, ABSCAL.HIS file format)

This repository was created to display the results of the absolute calibration with liquid nitrogen of the microwave radiometer HATPRO manufactured by Radiometer Physics GmbH. For this purpose, the binary file ABSCAL.HIS is read in and converted into an xarray. Then the receiver gain, the temperature of the noise diode, the temperature of the system noise and the non-linearity factor are displayed in comparison to the previous and other prior calibrations.

Getting Started

Installation

eval_ac requires Python 3.9 or newer. The dependencies (numpy, xarray, matplotlib, netCDF4) are installed automatically.

1. Clone the repo

   git clone https://github.com/WillyWallace/eval_ac.git
   cd eval_ac
   

2. Install the package

   pip install .
   

   For development (editable install with test and lint tools):

   pip install -e ".[dev]"
   

(back to top)

Usage

Command line

eval-ac path/to/ABSCAL.HIS --netcdf abscal.nc

This writes the NetCDF file, the history plot results_ln2_cal.png, the drift plot results_ln2_drift.png and prints a short quality report of the latest calibration:

read 16 calibration entries from path/to/ABSCAL.HIS
16 of them are calibrations with liquid nitrogen
receiver 1: latest calibration 2026-04-29 (159 days ago)
receiver 2: latest calibration 2026-04-29 (159 days ago)
drift: all channels of the latest calibration are within the thresholds

Lines starting with warning: point to something that should be checked, see Quality checks.

python -m eval_ac ... works as well. Try it with the example file:

eval-ac example_data/ABSCAL.HIS

History plot

The plot shows the receiver gain, the noise diode temperature, the system noise temperature and the non-linearity factor of both receivers. The latest and the previous calibration of each receiver are highlighted with their dates, all older calibrations are drawn in grey. Channels that were not calibrated (calibration_flag = 0) are marked with a red cross. By default, only calibrations with liquid nitrogen are used (cal_type = 1); if only one receiver of an entry was calibrated with liquid nitrogen, only this receiver is used.

Drift plot

For each receiver, the latest calibration is compared with the median of the 5 calibrations before it (deviation in percent). Channels that were not calibrated are excluded. The dashed red lines are the thresholds, channels of the latest calibration exceeding them are circled and listed in the report. The grey lines show how far the older calibrations deviate from the same reference.

The y-axis is scaled to the thresholds and to the latest and previous calibration. Large outliers of older calibrations are therefore cut off at the edge of the panel; they are still visible in the history plot.

Quality checks

The report checks the latest calibration of each receiver (only calibrations with liquid nitrogen, unless --all-cal-types is given):

With --fail-on-warning, eval-ac exits with code 2 if any check gives a warning, e.g. to send an e-mail from a cron job:

eval-ac /data/ABSCAL.HIS --no-plot --fail-on-warning || echo "check calibration" | mail -s "HATPRO" me@example.org

Interpreting the results

The report and the drift plot point to calibrations that deserve a closer look; they do not decide whether a calibration is good or bad. That decision needs the knowledge of the instrument and of the calibration conditions. The background below is taken from the RPG manual Principle of Operation & Software (standard radiometers), RPG-MWR-STD-SW (cited as "manual, section …").

What the absolute calibration determines

During the absolute calibration the radiometer looks at the internal ambient target and at the external liquid nitrogen cooled target, each with and without additional noise from the noise diode. From these four measurements it determines four parameters per channel (manual, section 3.1.3.1):

Consequences for the evaluation

• Changes of T_n and α are the most relevant: until the next absolute calibration, every automatic calibration builds on them. A real change of T_n that is not captured by a new absolute calibration enters the measured brightness temperatures.
• Differences of the gain between absolute calibrations are expected, because the gain follows the receiver temperature and is recalibrated during operation anyway. Its default threshold is therefore larger. In the example data the gain varies much more than the other variables (standard deviation 1–3 % in the K-band and 5–18 % in the V-band, compared with below 2 % for T_n and T_sys).
• ABSCAL.HIS also contains successful sky tipping calibrations (manual, section 4.6). They use a different method and are only possible in transparent channels, so eval_ac compares only calibrations with liquid nitrogen by default.

Why the median of the previous calibrations?

A single failed calibration in the reference would shift a mean, but hardly the median. The previous calibration is part of the reference, so its deviation is usually small.

Typical patterns

These are rules of thumb, not statements of the manual:

What to do with a suspicious calibration

• Check that the absolute calibration was done after a warm-up of at least 30 minutes; it is recommended every 5 to 6 months and after transport (manual, section 3.1.3).
• Repeat the calibration if the conditions were doubtful.
• A bad calibration can be removed in the RPG host software: in the Absolute Calibration History menu, Delete Last Entries removes all entries after the marked one, and Generate a new calibration file creates an ABSCAL.CLB from selected calibrations of receiver 1 and 2 (manual, section 4.6).

Adapting the thresholds

The default thresholds are derived from a single HATPRO (2018–2026, 16 calibrations). Instruments, channels and sites differ, so:

1. run eval-ac on the full history of your instrument,
2. look at the spread of the grey lines in the drift plot,
3. set thresholds slightly above the usual spread, e.g. -t gain=6 -t temp_noise=1.5.

Limitations

• With fewer than 5 earlier calibrations, the reference consists of fewer values (see n_reference_used in the result of calibration_drift()); with none, no drift can be computed.
• The drift is relative to the previous calibrations. A slow drift over many calibrations shifts the reference as well and is better seen in the history plot.

Python

from eval_ac.convert_abscal_his import read_abscal_his, write_netcdf
from eval_ac.analysis import select_cal_type, calibration_drift, drift_exceedances
from eval_ac.plotting import plot_calibration_history, plot_drift

ds = read_abscal_his("ABSCAL.HIS")   # xarray.Dataset, all calibration types
write_netcdf(ds, "abscal.nc")

ln2 = select_cal_type(ds)            # only calibrations with liquid nitrogen
fig = plot_calibration_history(ln2, variables=["gain", "temp_sys"])
fig.savefig("gain_tsys.png")

drift = calibration_drift(ln2, n_reference=5)
for row in drift_exceedances(drift, thresholds={"gain": 8}):
    print(row)
plot_drift(drift, thresholds={"gain": 8}).savefig("drift.png")

The dataset has the dimensions n_samples (calibration entries, oldest first) and freq (channels). The coordinate time holds the date of each calibration, the coordinate receiver (1 or 2) assigns each channel to its receiver.

Running the tests

pip install -e ".[test]"
pytest

(back to top)

Roadmap

• add meaningful docstrings
• make documentation with Sphinx (docs/)
• host the documentation on readthedocs
• enable pip install ...
• publish on PyPI
• Released version 1
• Add Tests

See the open issues for a full list of proposed features (and known issues) and CHANGELOG.md for the changes of each version.

(back to top)

License

Distributed under the MIT License. See LICENSE for more information.

(back to top)

Contact

Andreas Foth

(back to top)

Acknowledgments

Special thanks for templates and help during implementation.

• Readme Template
• cloudnetpy GitHub

(back to top)