no_mersi

Status

What works:

• Processing FY3D and FY3F raw data to l1b for 1km and 250m data
• Generating the associated GEO files, navigation is using both GPS data and TLE if possible. Navigation has not been carefully validated.
• CDF-based destriping for all TEB and RSB channels (see below)

What doesn’t (but we will work on it at some point):

• The thermal band calibration is very bad

Installation

For now, the best is to clone the repo, and then use uv run as show below.

Usage

uv run fy3-raw-to-l1b data/fy3d/station/rawdata_FY3D_MPT_f1_ch1_20260320T115601Z_20260320T121022Z_20260320T115601Z_zP4P11Ur3_vcdu2_43240.data --compression blosc generated --tle-file data/tle/tle-202603201215.txt

this will put the l1b and navigation files in the generated directory

I you want on visualise the results, you can use eg Satpy to generated some imagery.

For example, with:

import hdf5plugin  # noqa, for blosc decompression
from satpy import Scene
scn = Scene(["generated/FY3D_20260320_115604_121013_43240_MERSI_1000M_L1B.HDF", 
             "generated/FY3D_20260320_115604_121013_43240_MERSI_GEO1K_L1B.HDF"], reader="mersi2_l1b")

comp = "true_color_raw"
comps = [comp]
scn.load(comps)
scn.show(comp)

lscn = scn.resample("euron1")
lscn.show(comp)

Destriping

Destriping is enabled by default for all channels (TEB CH20–CH25 and RSB CH01–CH19). It uses CDF (Cumulative Distribution Function) matching: the radiance histogram of each detector is compared to the ensemble of all healthy detectors, and a Look-Up Table (LUT) is derived to equalise the distributions.

LUT persistence

LUTs are derived from cumulative histogram statistics that are stored between runs:

~/.local/share/no_mersi/destripe/
    fy3d_CH24_hist.npz      ← per-channel histogram state
    fy3d_CH24_ledger.json   ← deduplication log (prevents double-counting a pass)
    ...

After each processed granule the histograms are updated via an Exponential Moving Average (α = 0.05), so the correction improves progressively as more passes accumulate.

First run — bootstrap

On the first run for a given channel there is no prior histogram, so the processor performs an extra pre-pass through the full granule to build the initial LUT. This makes the first run noticeably slower (roughly double the usual processing time). Subsequent runs load the persisted state and are not affected.

Corrections (on by default) and how to skip them

The cleanest imagery is produced with no extra flags — CDF destriping, the RVS scene correction, DCN (8 column bins), the per-detector non-linearity correction, and an ADMM anisotropic-TV residual destripe (FY3D CH24/CH25; FY3F CH23/CH24/CH25) are all applied by default. Opt out individually:

uv run fy3-raw-to-l1b ... --no-destripe          # skip CDF destriping
uv run fy3-raw-to-l1b ... --no-dcn-correction    # skip DCN (fastest path)
uv run fy3-raw-to-l1b ... --no-rvs               # skip RVS scene correction
uv run fy3-raw-to-l1b ... --no-nonlinearity      # skip the non-linearity polynomial
uv run fy3-raw-to-l1b ... --no-admm-destripe     # skip the ADMM residual destripe (TEB)

--all-corrections is accepted as a legacy no-op alias (everything is already default-on).

To reset the destripe LUT history for a specific channel (e.g. after an instrument anomaly), delete the corresponding .npz and .json files in the XDG data directory above.