rail_cmnn

RAIL interface to Melissa Graham's CMNN algorithm. A slight modification of the original code found here:
dirac-institute/CMNN_Photoz_estimator

See https://ui.adsabs.harvard.edu/abs/2018AJ....155....1G/abstract for more details on the code Any use of rail_cmnn in a paper or report should cite Graham et al. (2018).

The current version of the code consists of a training stage, CMNNInformer, that computes colors for a set of training data and an estimation stage CMNNEstimator that calculates the Mahalanobis distance to each training galaxy for each test galaxy. The mean value of this Guassian PDF can be estimated in one of three ways (see selection mode below), and the width is determined by the standard deviation of training galaxy redshifts within the threshold Mahalanobis distance. Future implementation improvements may change the output format to include multiple Gaussians.

For the color calculation, there is an option for how to treat the "non-detections" in a band: the default choice is to ignore any colors that contain a non-detect magnitude and adjust the number of degrees of freedom in the Mahalanobis distance accordingly (this is how the CMNN algorithm was originally implemented). Or, if the configuration parameter nondetect_replace is set to True in CMNNInformer, the non-detected magnitudes will be replaced with the 1-sigma limiting magnitude in each band as supplied by the user via the mag_limits configuration parameter (or by the default 1-sigma limits if the user does not supply specific numbers). We have not done any exploration of the relative performance of these two choices, but note that there is not a significant performance difference in terms of runtime between the two methods.

CMNNInformer takes in a training data set and returns a model file that simply consists of the computed colors and color errors (magnitude errors added in quadrature) for that dataset, the model to be used in the CMNNEstimator stage. A modification of the original CMNN algorithm, "nondetections" are now replaced by the 1-sigma limiting magnitudes and the non-detect magnitude errors replaced with a value of 1.0. The config parameters that can be set by the user for CMNNInformer are:

• bands: list of the band names that should be present in the input data.
  
• err_bands: list of the magnitude error column names that should be present in the input data.
  
• redshift_col: a string giving the name for the redshift column present in the input data.
  
• mag_limits: a dictionary with keys that match those in bands and a float with the 1 sigma limiting magnitude for each band.
  
• nondetect_val: float or np.nan, the value indicating a non-detection, which will be replaced by the values in mag_limits.
  
• nondetect_replace: bool, if set to False (the default) this option ignores colors with non-detected values in the Mahalanobis distance calculation, with a corresponding drop in the degrees of freedom value. If set to True, the method will replace non-detections with the 1-sigma limiting magnitudes specified via mag_limits (or default 1-sigma limits if not supplied), and will use all colors in the Mahalanobis distance calculation.

The parameters that can be set via the config_params in CMNNEstimator are described in brief below:

• bands, err_bands, redshift_col, mag_limits are all the same as described above for CMNNInformer.
  
• ppf_value: float, usually 0.68 or 0.95, which sets the value of the PPF used in the Mahalanobis distance calculation.
  
• selection_mode: int, selects how the central value of the Gaussian PDF is calculated in the algorithm, if set to 0 randomly chooses from set within the Mahalanobis distance, if set to 1 chooses the nearest neighbor point, if set to 2 adds a distance weight to the random choice.
  
• min_n: int, the minimum number of training galaxies to use.
  
• min_thresh: float, the minimum threshold cutoff. Values smaller than this threshold value will be ignored.
  
• min_dist: float, the minimum Mahalanobis distance. Values smaller than this will be ignored.
  
• bad_redshift_val: float, in the unlikely case that there are not enough training galaxies, this central redshift will be assigned to galaxies.
  
• bad_redshift_err: float, in the unlikely case that there are not enough training galaxies, this Gaussian width will be assigned to galaxies.
  

RAIL: Redshift Assessment Infrastructure Layers

RAIL is a flexible software library providing tools to produce at-scale photometric redshift data products, including uncertainties and summary statistics, and stress-test them under realistically complex systematics. A detailed description of RAIL's modular structure is available in the Overview on ReadTheDocs.

RAIL serves as the infrastructure supporting many extragalactic applications of the Legacy Survey of Space and Time (LSST) on the Vera C. Rubin Observatory, including Rubin-wide commissioning activities. RAIL was initiated by the Photometric Redshifts (PZ) Working Group (WG) of the LSST Dark Energy Science Collaboration (DESC) as a result of the lessons learned from the Data Challenge 1 (DC1) experiment to enable the PZ WG Deliverables in the LSST-DESC Science Roadmap (see Sec. 5.18), aiming to guide the selection and implementation of redshift estimators in DESC analysis pipelines. RAIL is developed and maintained by a diverse team comprising DESC Pipeline Scientists (PSs), international in-kind contributors, LSST Interdisciplinary Collaboration for Computing (LINCC) Frameworks software engineers, and other volunteers, but all are welcome to join the team regardless of LSST data rights.

Installation

Installation instructions are available under Installation on ReadTheDocs.

Contributing

The greatest strength of RAIL is its extensibility; those interested in contributing to RAIL should start by consulting the Contributing guidelines on ReadTheDocs.

Citing RAIL

RAIL is open source and may be used according to the terms of its LICENSE (BSD 3-Clause). If you used RAIL in your study, please cite this repository https://github.com/LSSTDESC/RAIL, and RAIL Team et al. (2025) https://arxiv.org/abs/2505.02928

@ARTICLE{2025arXiv250502928T,
       author = {{The RAIL Team} and {van den Busch}, Jan Luca and {Charles}, Eric and {Cohen-Tanugi}, Johann and {Crafford}, Alice and {Crenshaw}, John Franklin and {Dagoret}, Sylvie and {De-Santiago}, Josue and {De Vicente}, Juan and {Hang}, Qianjun and {Joachimi}, Benjamin and {Joudaki}, Shahab and {Bryce Kalmbach}, J. and {Kannawadi}, Arun and {Liang}, Shuang and {Lynn}, Olivia and {Malz}, Alex I. and {Mandelbaum}, Rachel and {Merz}, Grant and {Moskowitz}, Irene and {Oldag}, Drew and {Ruiz-Zapatero}, Jaime and {Rahman}, Mubdi and {Rau}, Markus M. and {Schmidt}, Samuel J. and {Scora}, Jennifer and {Shirley}, Raphael and {St{\"o}lzner}, Benjamin and {Toribio San Cipriano}, Laura and {Tortorelli}, Luca and {Yan}, Ziang and {Zhang}, Tianqing and {the Dark Energy Science Collaboration}},
        title = "{Redshift Assessment Infrastructure Layers (RAIL): Rubin-era photometric redshift stress-testing and at-scale production}",
      journal = {arXiv e-prints},
     keywords = {Instrumentation and Methods for Astrophysics, Cosmology and Nongalactic Astrophysics, Astrophysics of Galaxies},
         year = 2025,
        month = may,
          eid = {arXiv:2505.02928},
        pages = {arXiv:2505.02928},
          doi = {10.48550/arXiv.2505.02928},
archivePrefix = {arXiv},
       eprint = {2505.02928},
 primaryClass = {astro-ph.IM},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2025arXiv250502928T},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

Please consider also inviting the developers as co-authors on publications resulting from your use of RAIL by making an issue. A convenient list of what to cite may be found under Citing RAIL on ReadTheDocs. Additionally, several of the codes accessible through the RAIL ecosystem must be cited if used in a publication.