shachen (沙尘)

Infrared satellite dust storm detection in Python — an open implementation of DEBRA-Dust, the Dynamic Enhancement with Background Reduction Algorithm (Miller et al. 2017, doi:10.1002/2017JD027365), for GOES ABI and Himawari AHI.

shachen (沙尘) is Chinese for "sand and dust". The package is a home for infrared-channel dust algorithms.

This appears to be the first public implementation of DEBRA.

All equations follow the erratum published 26 February 2020, not the figures as originally printed.

Dust is the yellow modulation; everything else stays in greyscale infrared. Left is the case from the paper's Figure 6. Both panels come straight out of scripts/run_case.py, one per sensor.

What it does

DEBRA turns the split-window infrared signal that is specific to mineral dust into a per-pixel confidence field, by comparing each pixel against a dynamically estimated clear-sky background rather than a fixed threshold. That background is what lets it work over bright, emissivity-heterogeneous desert surfaces where fixed thresholds produce false alarms.

io.satellite.load_scene   L1b → bt_* / refl_* on the 2 km fixed grid (satpy)
        │
        ▼
pipeline.run_debra
├─ geo.regrid_latlon      MERRA-2 / CAMEL → satellite grid
├─ solar                  per-pixel solar zenith (day / twilight / night mix)
├─ background             scheme A: CAMEL emissivity × Planck(MERRA-2 skin T)
│    or composite         scheme B: 14-day cloud-cleared same-hour composite
├─ cloudmask              Eqs. 1–12, including the dust restoral term
├─ dust_tests             DT1–DT3, Eqs. 13–15, normalised per-pixel
├─ confidence             Eqs. 16–22 → cf_comb
├─ imagery                Eqs. 23–29, CF-modulated RGB
└─ render                 georeferenced PNG with coastlines (cartopy)

Only one background scheme is used at a time: run_debra requires exactly one of emissivity= or background=. The composite scheme carries the split-window water-vapour depression that the semi-analytic one lacks.

The second algorithm is the baseline the first is judged against: pipeline.run_dust_rgb is the classic Dust RGB (Lensky and Rosenfeld 2008; GOES-R Quick Guide) — three fixed infrared stretches, no background, no cloud mask. It needs no ancillary data, reads one band DEBRA does not (11.2 µm), and returns the same (y, x, gun) layout, so both render through the same path. scripts/run_case.py writes it beside every DEBRA image for comparison.

Its stretches are picked per sensor from the scene's reader — the scheme has no single canonical set of numbers, having been re-tuned for each imager after SEVIRI — so the baseline is that satellite's operational product rather than a recipe borrowed from another one. The Dust RGB page has the table and the references.

Documentation

https://ringsaturn.github.io/shachen/ — user guide, all 29 equations as implemented, full API reference, and the deviations page.

make -C docs html      # → docs/_build/html/index.html
make -C docs latexpdf  # → docs/_build/latex/shachen.pdf (needs a TeX install)

Install

The core install is small — it runs the entire algorithm on fields you already have in memory, and pulls in no I/O or plotting stack:

pip install shachen

Add what you need on top:

pip install "shachen[satellite]"   # satpy: read and calibrate ABI/AHI L1b
pip install "shachen[data]"        # earthaccess/s3fs: fetch MERRA-2, CAMEL, L1b
pip install "shachen[render]"      # cartopy/matplotlib: georeferenced PNG
pip install "shachen[all]"         # everything, for the reproduction scripts

import shachen pulls in none of the extras.

Usage

import shachen

result = shachen.run_debra(scene, emissivity=camel, skin_temperature=merra_ts)
result["cf_comb"]  # combined dust confidence, 0–1

baseline = shachen.run_dust_rgb(scene)  # the classic Dust RGB, for comparison
baseline["dust_rgb"]  # (y, x, gun) floats in 0–1

What scene must contain, the two background schemes, and the imagery chain are covered in the user guide.

Reproducing a reference case end to end needs [all] plus an Earthdata login in ~/.netrc (MERRA-2 and CAMEL are authenticated downloads; GOES L1b on AWS S3 is anonymous):

python scripts/fetch_case.py 2017-03-23-swus   # the paper's Figure 6 case
python scripts/run_case.py   2017-03-23-swus   # → netCDF + PNG

Deviations from the paper

Three printed equations are inconsistent with the paper's own prose and figures even after the erratum, and are implemented per the prose:

Plus one substitution (CAMEL emissivity for the registration-walled UWBF) and one opt-in per-sensor retune.

All of it, with the reasoning and the numbers, is in docs/deviations.md. Read that before changing any of it. constants.py is the single source of every calibration bound, offset and weight from the paper, unit-tested against an independent transcription.

Citation

If you use this software, please cite the original algorithm:

Miller, S. D., Bankert, R. L., Grasso, L. D., Lindsey, D. T., Kuciauskas, A. P., & Combs, C. L. (2017). A dynamic enhancement with background reduction algorithm: Overview and application to satellite-based dust storm detection. Journal of Geophysical Research: Atmospheres, 122, 12,938–12,959. https://doi.org/10.1002/2017JD027365

and, for the implementation, the metadata in CITATION.cff.

This is an independent implementation. It is not produced, endorsed, or verified by the paper's authors, by CIRA, or by NOAA.

License

Apache-2.0. See NOTICE for attribution requirements.