XCheck: A Consistency-Based Validation Framework for Englacial Layer Annotations

This python package repository provides the implementation for "XCheck: A Consistency-Based Validation Framework for Englacial Layer Annotations" (Paper), accepted for publication at IEEE ICDMW 2025 proceedings. We introduce a dot-product-based layer consistency framework that validates annotated englacial layers between two radargrams observing the same ice volume — either two consecutive flight segments (overlap detected via GPS time) or two intersecting flight paths (crossing point supplied via a pre-computed CSV).

Introduction

Ice-penetrating radar surveys produce radargrams — 2-D depth profiles of englacial structure. Human or automated annotators trace internal reflection horizons (layers) that represent isochrones of ice deposition. When two flight lines observe the same ice at a crossing or overlap point, consistent annotations should agree in depth. XCheck operationalizes this constraint: it extracts binary depth masks from annotated layers, aligns them geometrically at the shared observation point, and uses a sliding dot-product test to find matching layer pairs. A high match count indicates annotation consistency; a low count flags potential mislabelling or pick drift.

XCheck handles two radargram relationship types:

• Consecutive — two sequential segments of the same flight line share a tail/head overlap detectable via GPS timestamp matching.
• Non-consecutive (intersecting) — two flight lines from different dates or paths cross at a known geographic point supplied by a pre-computed intersections CSV.

Methodology

Algorithm Steps

Requires: Binary Masks M1, M2 · GPS time sequences T1, T2 · Flight paths P1, P2 · Surface elevations S1, S2 · Depth axis D · Window size w

Installation

pip install xcheck

NDH_PythonTools (required for data loading)

XCheck uses NDH_PythonTools for .mat file loading and radar data processing. It must be installed manually since it is not a standard pip package:

git clone https://github.com/nholschuh/NDH_PythonTools.git

Add its parent directory to PYTHONPATH before running:

# Linux / macOS
export PYTHONPATH="/path/to/parent/of/NDH_PythonTools"

# Windows PowerShell
$env:PYTHONPATH = "C:\path\to\parent\of\NDH_PythonTools"

The pure matching algorithm (match_layers_with_dot_product) does not require NDH_PythonTools and works standalone on any NumPy arrays.

Directory Structure

pypi package/
├── src/xcheck/
│   ├── xcheck.py          # Core algorithms: loading, masking, GPS overlap,
│   │                      #   column extraction, layer matching, plotting, CLI
│   └── __init__.py        # Public API exports
├── pyproject.toml         # Package metadata and dependencies
├── README.md
└── dist/                  # Built wheel and source distribution

Quickstart

1. Command Line Interface

Consecutive radargrams (crossing derived automatically from GPS overlap):

xcheck --r1 Data_20120429_01_026.mat --r2 Data_20120429_01_027.mat \
       --layer_dir ./Nick-layer-data-mat/ \
       --radar_dir ./Nick-raw-radargram-mat/

Intersecting radargrams (crossing looked up from CSV):

xcheck --r1 Data_20120330_01_004.mat --r2 Data_20120511_01_054.mat \
       --intersections_csv ./Intersections_2012.csv \
       --layer_dir ./Nick-layer-data-mat/ \
       --radar_dir ./Nick-raw-radargram-mat/

Batch mode (all pairs in the intersections CSV):

xcheck --intersections_csv ./Intersections_2012.csv \
       --layer_dir ./Nick-layer-data-mat/ \
       --radar_dir ./Nick-raw-radargram-mat/

2. Python API

Layer matching only (no NDH_PythonTools needed):

import numpy as np
from xcheck import match_layers_with_dot_product

m1 = np.zeros((6, 5)); m1[[1, 3, 5], :] = 1
m2 = np.zeros((6, 5)); m2[[1, 3, 5], :] = 1

matches = match_layers_with_dot_product(m1, m2, max_distance=1,
                                        window_size=3, dp_threshold=1)
print(matches)  # [(1, 1), (3, 3), (5, 5)]

Full pipeline (requires NDH_PythonTools + data):

from xcheck import (
    load_and_process_radargram,
    load_log_gps_time,
    find_overlapping_intervals,
    extract_and_adjust_mask_columns_for_location,
    match_layers_with_dot_product,
)

r1, m1, d1 = load_and_process_radargram(
    "Data_20120429_01_026.mat",
    layer_dir="./Nick-layer-data-mat/",
    radar_dir="./Nick-raw-radargram-mat/")
r2, m2, d2 = load_and_process_radargram(
    "Data_20120429_01_027.mat",
    layer_dir="./Nick-layer-data-mat/",
    radar_dir="./Nick-raw-radargram-mat/")

log_t1 = load_log_gps_time("./Nick-raw-radargram-mat/Data_20120429_01_026.mat")
log_t2 = load_log_gps_time("./Nick-raw-radargram-mat/Data_20120429_01_027.mat")
overlaps1, overlaps2 = find_overlapping_intervals(log_t1, log_t2)

for iv1, iv2 in zip(overlaps1, overlaps2):
    mid1 = (iv1[0] + iv1[1]) // 2
    mid2 = (iv2[0] + iv2[1]) // 2
    mc1 = extract_and_adjust_mask_columns_for_location(
        r1['Longitude'][mid1], r1['Latitude'][mid1], r1, m1, d1, 20)
    mc2 = extract_and_adjust_mask_columns_for_location(
        r2['Longitude'][mid2], r2['Latitude'][mid2], r2, m2, d2, 20)
    matches = match_layers_with_dot_product(mc1, mc2)
    print(f"Matched layers: {matches}")

Parameters

Defaults shown as consecutive / non-consecutive — selected automatically based on whether a crossing point is available.

Default Thresholds by Radargram Type

Methodological Note

GPS overlap detection compares log(GPS_time) between two radargrams with an absolute tolerance of 1e-8. Because d(log t) ≈ dt/t and GPS times are ~1×10⁹ seconds, this tolerance corresponds to matching raw timestamps within roughly 10 seconds. Consecutive flight segments share a small tail/head of overlapping traces where the crossing point is derived. Non-consecutive intersecting radargrams do not share GPS timestamps — their crossing point must be supplied via --intersections_csv or --lat/--lon.

Citation

Cite our work.

@INPROCEEDINGS{11415920,
  author={Hassan, Muhammad Behroze and Tama, Bayu Adhi and Purushotham, Sanjay and Janeja, Vandana P.},
  booktitle={2025 IEEE International Conference on Data Mining Workshops (ICDMW)}, 
  title={XCheck: A Consistency-Based Validation Framework for Englacial Layers Annotations}, 
  year={2025},
  volume={},
  number={},
  pages={76-85},
  keywords={Surveys;Radar remote sensing;Annotations;Ice sheets;Radar;Benchmark testing;Radar tracking;Robustness;Data mining;Remote sensing;Ice sheets;englacial layers;radargrams spatiotemporal relationship;validation metric;AI-ready dataset},
  doi={10.1109/ICDMW69685.2025.00015}
}

References

M. B. Hassan, B. A. Tama, S. Purushotham and V. P. Janeja, "XCheck: A Consistency-Based Validation Framework for Englacial Layers Annotations," 2025 IEEE International Conference on Data Mining Workshops (ICDMW), Washington, DC, USA, 2025, pp. 76-85, doi: 10.1109/ICDMW69685.2025.00015.

License

MIT