specsrbench

Benchmarking deep learning against classical deconvolution for galaxy spectral super-resolution.

Every method in the benchmark, on a 1-D toy where the truth is known: a close doublet the instrument blends into one blob, and a weak isolated line. Drawn by scripts/make_classical_toy_figure.py using the same deconvolvers the benchmark runs.

Seven classical deconvolution methods — cubic interpolation, Wiener, Tikhonov, Wiener + total variation, Richardson–Lucy, wavelet-sparse FISTA, and a redshift-informed matched filter — scored against the SR2 deep-learning pipeline of Haghjoo et al. 2026 on JWST/NIRSpec prism spectra from JADES.

The deep-learning side is a separate package, specsr, whose weights live on the Hub. This package holds the classical baselines, the metrics, the tuning, and the figures.

Install

pip install specsrbench          # the baselines and the figures
pip install 'specsrbench[all]'   # + the ML arm and the LSF derivation

Or from source, for development:

git clone https://github.com/aryana-haghjoo/specsr-benchmark
cd specsr-benchmark
pip install -e '.[dev]'

[ml] pulls in torch and specsr (needed by build predictions and by figure 1's toy CNN); [lsf] pulls in astropy (needed only to re-derive the instrument LSF). Neither is required to draw the other five figures.

What it is for

The short version of the result: on this data, at the pixel level, no method beats cubic interpolation.

SR2 leads the raw mean-absolute-error table by 30%, and does it by producing a spectrum at 0.54 of the reference's amplitude — absolute error against a noisy reference falls when you shrink toward zero, whatever the reconstruction quality. On a scale-free metric all eight methods land within 1.2% of each other and SR2 ranks seventh of eight. What does survive is sharper and more interesting than a leaderboard: SR2 exceeds the reference's own line signal to noise on all four diagnostic lines while recovering only 36–53% of true line amplitudes, at false-detection rates of 0.30 (Hβ) and 0.44 ([O II]) against ≤ 0.09 for any classical method.

Line flux ratios fare no better. Measured from integrated Gaussian fluxes, with Hβ and [O III] fitted jointly, no method improves on the interpolated prism spectrum; SR2's ratios are unbiased in the median but scatter about twice as widely as the reference's.

Getting there took four separate metric traps, each of which had to be closed before the benchmark meant anything. They are documented in docs/GUARDS.md, and they are the part of this repository most likely to be useful to someone benchmarking something else.

Start here: the tutorial

If you want to see what this does before installing survey data, run the notebook. It needs no JADES tree, no GPU and no configuration — 24 held-out spectra download from the Hub on first use and everything after that is numpy and scipy.

pip install specsrbench          # enough: fetching the sample is built in
jupyter lab tutorials_for_user/01_quickstart.ipynb

It runs all six classical deconvolvers at the tuned parameters, scores them against the grating reference alongside the deep-learning prediction, and then demonstrates live why the obvious metric reverses the result — see tutorials_for_user/.

from specsrbench.sample import load_sample

s = load_sample()          # 1.9 MB, cached after the first call
recon = s.reconstruct()    # all six classical methods, a few seconds

Quick start

specsrbench paths           # where it will look for inputs
specsrbench figures all     # rebuild all six paper figures
specsrbench figures 4       # or just one

The figures read a cache of derived arrays. If you do not have one, build it (next section) or point at one you do have:

export SPECSRBENCH_CACHE=/path/to/cache_logR_tuned

Figure 1 trains a small CNN inline and needs specsrbench[ml]; the other five are pure numpy and matplotlib.

Rebuilding everything from the data

No large files are distributed with this package. The chain runs from the raw JADES DR4 products and the published model weights, in six stages:

specsrbench build predictions   # specsr chain over the held-out split (Hub weights)
specsrbench build sets          # eval / tune / calib sets, galaxy-disjoint
specsrbench build lsf --jades-root <JADES DR4>   # measure the instrument LSF
specsrbench build tune          # search the classical parameters under four guards
specsrbench build classical     # the classical reconstructions
specsrbench build lines         # Gaussian line fits, S/N, line fluxes, the summary table

specsrbench build all runs them in order. --dry-run on any stage prints what it would read and write without doing it.

What each stage needs:

The paired dataset (3.5 GB) is built by specsr from the JADES tree and is not redistributable here:

specsr build --jades-root <JADES DR4 tree> --out paired_DR4_logR.npz

The one thing to get right

Match the checkpoint to the wavelength grid by reading its provenance, never by its filename. More than one file called best_sr2.pth exists, trained on different grids. Loading the wrong one does not fail — it produces plausible spectra that are wrong. specsrbench build predictions pins the Hub revision and records what it loaded in the output's provenance field.

Why the LSF is measured and not read

The evaluation products carry a sigma_pix array that was widely used as the deconvolution kernel and does not describe the data: it is roughly constant in nanometres, where a spectrograph's line-spread function is fixed in detector pixels. Measured against the paired spectra it is up to 2.3× too broad at 5 µm. With it, Wiener, Tikhonov and TV merge line pairs that their own input still resolves.

specsrbench build lsf measures the real one from the instrument: the prism dispersion from the raw x1d products, the effective kernel width from σ_eff² = σ_LR² − σ_HR² on fits to the four diagnostic lines, and the ratio of the two. It comes out constant to a few per cent across a factor of three in wavelength, which is the check that it is an instrumental LSF and not a curve fitted to noise. The sets this package builds deliberately do not carry a sigma_pix, so the kernel can only come from measuring it.

Tuning is guarded, in four ways

Mean absolute error is not a safe objective, and no single guard catches what the others do:

Each is blind to the others: line S/N is amplitude over sideband noise, so a global rescale leaves it unchanged; the amplitude guard is width-invariant; and all three of those are single-line statistics, so a Gaussian fitted to a blended doublet passes them. The fourth is measured on real spectra — a synthetic pair was tried and discarded as far too easy to hold apart.

specsrbench build tune applies all four inside the search, and tests/test_invariants.py asserts them against whatever cache is present. A method is allowed to fail guard 4 only if classical_params.json records that no setting passes it; that is Tikhonov's case, and the paper says so.

Tests

pytest                    # the full suite
pytest -m "not slow"      # skip the end-to-end figure builds

The suite runs without any data — tests that need a cache skip cleanly. With one present it checks the cache's structural invariants, the four guards, and rebuilds a slice of each classical cache from the recorded parameters to confirm it reproduces.

Layout

src/specsrbench/
  paths.py      where inputs and outputs are found
  methods.py    the eight methods and the reference, their names and cache keys
  metrics.py    MAE, scale-free MAE, amplitude ratio, the guards
  classical.py  the seven classical deconvolvers, on the log R=4000 grid
  data.py       the cache loader every figure shares
  figures/      one module per paper figure
  build/        the six pipeline stages
  sample.py     the 24-spectrum tutorial sample, fetched from the Hub
scripts/        talk figures, which are not paper figures
tutorials_for_user/  the tutorial notebook
tests/
figures/        where `specsrbench figures all` writes its PDFs

Documentation

Full documentation, including the API reference, is at https://aryana-haghjoo.github.io/specsr-benchmark/, and the tutorial has its own page.

The tutorial sample is hosted separately, on the Hub: https://huggingface.co/datasets/aryana-haghjoo/specsr-benchmark.

Citing

Paper 2 is in preparation. Until it appears, cite the archived software release and Haghjoo et al. 2026 for the models.

Every release is archived on Zenodo:

• 10.5281/zenodo.22104971 — cite this one unless you need a specific version. It is the concept DOI and always resolves to the latest release.
• 10.5281/zenodo.22105099 — v0.1.1 specifically, for when the exact version matters (a result you are reproducing, say).

CITATION.cff carries the machine-readable metadata; GitHub's "Cite this repository" button reads it.

License

MIT. See LICENSE.