pyMCR 0.3.3

Multivariate Curve Resolution in Python

pyMCR: Multivariate Curve Resolution in Python

Documentation available online at https://pages.nist.gov/pyMCR

Software DOI: https://doi.org/10.18434/M32064

Manuscript DOI: https://doi.org/10.6028/jres.124.018

pyMCR is a small package for performing multivariate curve resolution. Currently, it implements a simple alternating regression scheme (MCR-AR). The most common implementation is with ordinary least-squares regression, MCR-ALS.

MCR with non-negativity constraints on both matrices is the same as non-negative matrix factorization (NMF). Historically, other names were used for MCR as well:

• Self modeling mixture analysis (SMMA)

• Self modeling curve resolution (SMCR)

Available methods:

• Regressors:

  • Ordinary least squares (default)

  • Non-negatively constrained least squares

  • Native support for scikit-learn linear model regressors (e.g., LinearRegression, RidgeRegression, Lasso)

• Constraints

  • Non-negativity

  • Normalization

  • Zero end-points

  • Zero (approx) end-points of cumulative summation (can specify nodes as well)

  • Non-negativity of cumulative summation

  • Compress or cut values above or below a threshold value

  • Replace sum-across-features samples (e.g., 0 concentration) with prescribed target

  • Enforce a plane (“planarize”). E.g., a concentration image is a plane.

• Error metrics / Loss function

  • Mean-squared error

• Other options

  • Fix known targets (C and/or ST, and let others vary)

What it does do:

• Approximate the concentration and spectral matrices via minimization routines. This is the core the MCR methods.

• Enable the application of certain constraints in a user-defined order.

What it does not do:

• Estimate the number of components in the sample. This is a bonus feature in some more-advanced MCR-ALS packages.

  • In MATLAB: https://mcrals.wordpress.com/

  • In R: https://cran.r-project.org/web/packages/ALS/index.html

Dependencies

Note: These are the developmental system specs. Older versions of certain packages may work.

• python >= 3.4

  • Tested with 3.4.6, 3.5.4, 3.6.3, 3.6.5, 3.7.1

• numpy (1.9.3)

  • Tested with 1.12.1, 1.13.1, 1.13.3, 1.14.3, 1.14.6

• scipy (1.0.0)

  • Tested with 1.0.0, 1.0.1, 1.1.0

• scikit-learn, optional (0.2.0)

Known Issues

Installation

Using pip (hard install)

# Only Python 3.* installed
pip install pyMCR

# If you have both Python 2.* and 3.* you may need
pip3 install pyMCR

Using pip (soft install [can update with git])

# Make new directory for pyMCR and enter it
# Clone from github
git clone https://github.com/usnistgov/pyMCR

# Only Python 3.* installed
pip install -e .

# If you have both Python 2.* and 3.* you may need instead
pip3 install -e .

# To update in the future
git pull

Using setuptools

You will need to download the repository or clone the repository with git:

# Make new directory for pyMCR and enter it
# Clone from github
git clone https://github.com/usnistgov/pyMCR

Perform the install:

python setup.py install

Logging

New in pyMCR 0.3.1, Python’s native logging module is now used to capture messages. Though this is not as convenient as print() statements, it has many advantages.

• Logging module docs: https://docs.python.org/3.7/library/logging.html

• Logging tutorial: https://docs.python.org/3.7/howto/logging.html#logging-basic-tutorial

• Logging cookbook: https://docs.python.org/3.7/howto/logging-cookbook.html#logging-cookbook

A simple example that prints simplified logging messages to the stdout (command line):

import sys
import logging

# Need to import pymcr or mcr prior to setting up the logger
from pymcr.mcr import McrAR

logger = logging.getLogger('pymcr')
logger.setLevel(logging.DEBUG)

# StdOut is a "stream"; thus, StreamHandler
stdout_handler = logging.StreamHandler(stream=sys.stdout)

# Set the message format. Simple and removing log level or date info
stdout_format = logging.Formatter('%(message)s')  # Just a basic message akin to print statements
stdout_handler.setFormatter(stdout_format)

logger.addHandler(stdout_handler)

# Begin your code for pyMCR below

Usage

from pymcr.mcr import McrAR
mcrar = McrAR()

# MCR assumes a system of the form: D = CS^T
#
# Data that you will provide (hyperspectral context):
# D [n_pixels, n_frequencies]  # Hyperspectral image unraveled in space (2D)
#
# initial_spectra [n_components, n_frequencies]  ## S^T in the literature
# OR
# initial_conc [n_pixels, n_components]   ## C in the literature

# If you have an initial estimate of the spectra
mcrar.fit(D, ST=initial_spectra)

# Otherwise, if you have an initial estimate of the concentrations
mcrar.fit(D, C=initial_conc)

Example Results

Command line and Jupyter notebook examples are provided in the Examples/ folder. Examples of instantiating the McrAR class with different regressors available in the documentation .

From Examples/Demo.ipynb:

Citing this Software

If you use pyMCR, citing the following article is much appreciated:

• C. H. Camp Jr., “pyMCR: A Python Library for Multivariate Curve Resolution Analysis with Alternating Regression (MCR-AR)”, Journal of Research of National Institute of Standards and Technology 124, 1-10 (2019).

References

• W. H. Lawton and E. A. Sylvestre, “Self Modeling Curve Resolution”, Technometrics 13, 617–633 (1971).

• https://mcrals.wordpress.com/theory/

• J. Jaumot, R. Gargallo, A. de Juan, and R. Tauler, “A graphical user-friendly interface for MCR-ALS: a new tool for multivariate curve resolution in MATLAB”, Chemometrics and Intelligent Laboratory Systems 76, 101-110 (2005).

• J. Felten, H. Hall, J. Jaumot, R. Tauler, A. de Juan, and A. Gorzsás, “Vibrational spectroscopic image analysis of biological material using multivariate curve resolution–alternating least squares (MCR-ALS)”, Nature Protocols 10, 217-240 (2015).

LICENSE

This software was developed by employees of the National Institute of Standards and Technology (NIST), an agency of the Federal Government. Pursuant to title 17 United States Code Section 105, works of NIST employees are not subject to copyright protection in the United States and are considered to be in the public domain. Permission to freely use, copy, modify, and distribute this software and its documentation without fee is hereby granted, provided that this notice and disclaimer of warranty appears in all copies.

THE SOFTWARE IS PROVIDED ‘AS IS’ WITHOUT ANY WARRANTY OF ANY KIND, EITHER EXPRESSED, IMPLIED, OR STATUTORY, INCLUDING, BUT NOT LIMITED TO, ANY WARRANTY THAT THE SOFTWARE WILL CONFORM TO SPECIFICATIONS, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND FREEDOM FROM INFRINGEMENT, AND ANY WARRANTY THAT THE DOCUMENTATION WILL CONFORM TO THE SOFTWARE, OR ANY WARRANTY THAT THE SOFTWARE WILL BE ERROR FREE. IN NO EVENT SHALL NIST BE LIABLE FOR ANY DAMAGES, INCLUDING, BUT NOT LIMITED TO, DIRECT, INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES, ARISING OUT OF, RESULTING FROM, OR IN ANY WAY CONNECTED WITH THIS SOFTWARE, WHETHER OR NOT BASED UPON WARRANTY, CONTRACT, TORT, OR OTHERWISE, WHETHER OR NOT INJURY WAS SUSTAINED BY PERSONS OR PROPERTY OR OTHERWISE, AND WHETHER OR NOT LOSS WAS SUSTAINED FROM, OR AROSE OUT OF THE RESULTS OF, OR USE OF, THE SOFTWARE OR SERVICES PROVIDED HEREUNDER.

Contact

Charles H Camp Jr: [e-mail]

Contributors

• Charles H Camp Jr

• Charles Le Losq ([e-mail])

• Robert Kern ([e-mail])

• Joshua Taillon ([e-mail])