Atlas-Fit

Python tools to fit solar spectra with known atlantes for

• wavelength calibration
• continuum correction
• stray-light analysis
• effective FWHM computation
• chi^2 error computation of the atlas fit

Summary

This package provides two tools:

• bin/setup:
  This script is used to test and fine-tune the configuration and to inspect the fitting and analysis results with interactive plots.
• bin/run:
  This tool allows to amend the calibration results from the spectroflat library. The fitted dispersion can be applied to the OffsetMap (to include wavelength calibration).
  The straylight and continuum correction will be appended to the calibrated illumination pattern (soft flat).

Both tools can (and should) be run with the same configuration. Run the setup first to find the right configuration values and use the run script to apply the results.

Getting Started

Installation

We recommend the installation via pip install altas_fit.
You can also pull from the repository, but be aware that the main branch is not frozen at the latest stable release. Please check out a git tag to be in a defined and tested state.

Example

Verify the installation and get a feeling for how to use it with the working example. It is provided in the examples folder. You can run it with

$ bin/setup examples/config.yml examples/fine_range.yml

With your own data

Please refer to the QUICKSTART GUIDE for a brief introduction on how to create and test the config for your data.

Input/Output data

Input

This library works on the calibration data from spectroflat. It uses

input:
  corrected_frame: flat_fielded_flat.fits  # The calibrated flat field frame (state, slit, wavelength)
  soft_flat: illumination_pattern.fits     # The slowly varying illumination patter (or 'soft flat') (state, slit, wavelength)
  offset_map: offset_map.fits              # The measured shifts from the smile correction (i.e., spectroflat::OffsetMap)

Output

The generated output are amended versions of the offset map and illumination pattern.

The offsets in the amended OffsetMap contain the smile and wavelength correction, such that the science data can be interpolated to a uniform linear wavelength grid in one go. It further contains now a wavelength axis and wavelength keywords in the header.

The amended Illumination Pattern contains

• HDU 0: the original illumination pattern shifted to the wavelength calibrated position
• HDU 1: 2D continuum correction surface
• HDU 2: 2D gray straylight surface
• HDU 3: 1D wavelength axis You need to scale the straylight surface with the intensity mean prior to subtraction. To keep [DN] values after continuum correction divide the continuum correction surface by its mean value.

THe header contains WCS ready wavelength information. However, you might need to re-process and adjust this within your pipeline

Citation

When using this library please cite:

Hölken et al. "Spectroflat: A generic spectrum and flat-field calibration library for spectro-polarimetric data" (DOI: 10.1051/0004-6361/202348877)

@ARTICLE{spectroflat,
       author = {{H{\"o}lken}, J. and {Doerr}, H.-P. and {Feller}, A. and {Iglesias}, F.~A.},
        title = "{Spectroflat: A generic spectrum and flat-field calibration library for spectro-polarimetric data}",
      journal = {\aap},
     keywords = {instrumentation: polarimeters, instrumentation: spectrographs, methods: data analysis, methods: numerical},
         year = 2024,
        month = jul,
       volume = {687},
          eid = {A22},
        pages = {A22},
          doi = {10.1051/0004-6361/202348877},
}

Versioning

We follow the Semantic Versioning pattern. In short semantic versioning boils down to:

• PATCH 0.0.x level changes for implementation level detail changes, such as small bug fixes
• MINOR 0.x.0 level changes for any backwards compatible API changes, such as new functionality/features
• MAJOR x.0.0 level changes for backwards incompatible API and algorithm changes, such as changes that will break existing users code if they update and or change the outcome of the code notably.

Please find a list of relevant releases on the git tags and releases pages. Upcoming versions are described as milestones.

Original algorithm

The atlas_fit algorithm 1.X.X is described and evaluated in Hölken et al. (2024) "Spectroflat: A generic calibration library for spectro-polarimetric data". [see above]

Changes in 2.X.X:

We have re-implemented the line selection. Now it is no longer required to click on individual lines. Instead, the user can now define the range in the atlas and fine tune the automated selection if needed.

• [Braking change] The scripts have new name prepare->setup and amend_spectroflat->run.
• [Braking change] We have simplified the configuration by large. The following config entries are no longer suppoerted and must be removed from the config. Removed config entries are:
  • input:roi use input:correction_roi to define the to-be-corrected area
  • fitting:lowpass_filter no longer needed as the new pre-straighten algorithm is more robust.
  • fitting:high_quantile_continuum no longer needed as the new pre-straighten algorithm is more robust.
  • fitting:continuum_spline_smoothing no longer needed as the continuum extraction is now robust
  • fitting:continuum_points no longer needed as the continuum extraction is now robust
  • fitting:continuum_points no longer needed as the continuum extraction is now robust
  • fitting:dispersion_min_degree no longer needed as dispersion is now measured, not fitted
  • fitting:dispersion_max_degree no longer needed as dispersion is now measured, not fitted
• Line Selection Lines are now pre-selected by their bisector. If deviation is too large (e.g. asymmetry) the line is disregarded. This ensures better measurement of the wavelength solution
• Wavelength Solution The wavelength solution is now directly measured from the discrepancy of the line positions in atlas and data space. From this discrepancy shifts to a linear wavelength scale are computed.
• Continuum Correction We now directly fit a high order polynomial through the ration of atlas and data. The fit is weighted to follow the continuum points and not the lines.
• Non-global straylight The algorithm now supports gray straylight with a linear gradient or second order curvature. By default, the penalty on these terms is high to favor global straylight terms.
• WCS headers We now provide WCS conform header information on the wavelength calibration
• New config entries (A full description can be found in the API documentation):
  • input:straighten This was a command line switch before, and should have been a config item
  • fitting:continuum_trend_degree To be used if input:straighten is True
  • fitting:continuum_level To tune automatic selection of lines
  • fitting:max_bisector_nonuniformity To tune automatic selection of lines
  • fitting:straylight_gradient_penalty Penalty to fit non-global straylight
  • fitting:straylight_non_linearity_penalty Penalty to fit non-global non-linear straylight

Users

This routine is part of the SUNRISE-III/SUSI data reduction pipeline. It is actively used for the calibration of GST/FISS-SP data.

It has additionally been tested with GREGOR/GRIS, SST/TRIPPEL, ZIMPOL, and other datasets.

Support

Developed and maintained by hoelken@mps.mpg.de

Contributions by iglesias@mps.mpg.de and feller@mps.mpg.de.

Contributing

Any contribution that follows our Code of Conduct and that helps to advance the atlas_fit library is highly welcome.

Bug reports & feature requests are welcome via gitlab issues or email. Issues can either be opened via the web interface (needs access to the gitlab.gwdg.de) or send to the maintainer(s) email address(es) directly.

To ease bug fixing please provide a reproducible description and/or relevant testing data in bug reports. If provided data needs to be handled confidentially, please email the maintainer(s) directly.

Code contributions are welcome as merge or pull requests and will undergo a code review. Any provided code must satisfy the python code style (i.e., the stylecheck.bash needs to pass) and have at least 80 % test coverage. Development forks of the repository are explicitly recommended.

License

This code is developed and maintained at the Max Planck Institute for Solar System Research (MPS) Göttingen.

License: BSD clause-2, see LICENSE