ltsfit 5.0.20

LtsFit: Least Trimmed Squares Fitting

The LtsFit Package

Robust Linear Regression with Scatter in One or Two Dimensions

LtsFit is a Python implementation of the method described in Section 3.2 of Cappellari et al. (2013a) to perform very robust fits of lines or planes to data with errors in all coordinates, while allowing for possible intrinsic scatter. Outliers are iteratively clipped using the robust Least Trimmed Squares (LTS) technique by Rousseeuw & van Driessen (2006).

Contents

• The LtsFit Package

  • Attribution

  • Installation

  • Documentation

• lts_linefit

  • Purpose

  • Explanation

  • Calling Sequence

  • Input Parameters

  • Output Parameters

• lts_planefit

  • Purpose

  • Explanation

  • Calling Sequence

  • Input Parameters

  • Output Parameters

• License

• Changelog

Attribution

If you use this software for your research, please also cite Cappellari et al. (2013a) where the implementation was described. The BibTeX entry for the paper is:

@ARTICLE{Cappellari2013a,
    author = {{Cappellari}, M. and {Scott}, N. and {Alatalo}, K. and
        {Blitz}, L. and {Bois}, M. and {Bournaud}, F. and {Bureau}, M. and
        {Crocker}, A.~F. and {Davies}, R.~L. and {Davis}, T.~A. and {de Zeeuw},
        P.~T. and {Duc}, P.-A. and {Emsellem}, E. and {Khochfar}, S. and
        {Krajnovi{\'c}}, D. and {Kuntschner}, H. and {McDermid}, R.~M. and
        {Morganti}, R. and {Naab}, T. and {Oosterloo}, T. and {Sarzi}, M. and
        {Serra}, P. and {Weijmans}, A.-M. and {Young}, L.~M.},
    title = "{The ATLAS$^{3D}$ project - XV. Benchmark for early-type
        galaxies scaling relations from 260 dynamical models: mass-to-light
        ratio, dark matter, Fundamental Plane and Mass Plane}",
    journal = {MNRAS},
    eprint = {1208.3522},
    year = 2013,
    volume = 432,
    pages = {1709-1741},
    doi = {10.1093/mnras/stt562}
}

Installation

install with:

pip install ltsfit

Without writing access to the global site-packages directory, use:

pip install --user ltsfit

Documentation

See ltsfit/examples and the files docstrings. They are copied by pip within the global folder site-packages.

---

lts_linefit

Purpose

Best straight-line robust fit to data with errors in both coordinates while fitting for the intrinsic scatter. See Cappellari et al. (2013a, Sec.3.2)

Explanation

Linear Least-squares approximation in one-dimension (y = a + b*x), when both x and y data have errors and allowing for intrinsic scatter in the relation.

Outliers are iteratively clipped using the extremely robust FAST-LTS technique by Rousseeuw & van Driessen (2006) See also Rousseeuw (1987)

Calling Sequence

p = lts_linefit(x, y, sigx, sigy, clip=2.6, epsy=True, corr=True,
              frac=None, pivot=None, plot=True, text=True)

The output values are stored as attributes of “p”. See usage example at the bottom of this file.

Input Parameters

x, y:

vectors of size N with the measured values.

sigx, sigy:

vectors of size N with the 1sigma errors in x and y.

clip:

values deviating more than clip*sigma from the best fit are considered outliers and are excluded from the linear fit.

epsy:

if True, the intrinsic scatter is printed on the plot.

corr:

if True, the correlation coefficients are printed on the plot.

frac:

fractions of values to include in the LTS stage. Up to a fraction “frac” of the values can be outliers. One must have 0.5 < frac < 1 (default frac=0.5).

NOTE: Set frac=1, to turn off outliers detection.

label:

optional string label to create a legend outside the procedure.

pivot:

if this is not None, then lts_linefit fits the relation:

y = a + b*(x - pivot)

pivot is called x_0 in eq.(6) of Cappellari et al. (2013) Use of this keyword is strongly recommended and a suggested value is pivot ~ np.mean(x). This keyword is important to reduce the covariance between a and b.

plot:

if True a plot of the fit is produced.

text:

if True, the best fitting parameters are printed on the plot.

Output Parameters

The output values are stored as attributed of the lts_linefit class.

p.ab:

best fitting parameters [a, b]

p.ab_err:

1*sigma formal errors [a_err, b_err] on a and b.

p.mask:

boolean vector with the same size of x and y. It contains True for the elements of (x, y) which were included in the fit and False for the outliers which were automatically clipped.

p.rms:

rms = np.std(fit - y) beteween the data and the fitted relation.

p.sig_int:

intrinsic scatter around the linear relation. sig_int is called epsilon_y in eq.(6) of Cappellari et al. (2013).

p.sig_int_err:

1*sigma formal error on sig_int.

---

lts_planefit

Purpose

Best plane robust fit to data with errors in all three coordinates and fitting for the intrinsic scatter. See Cappellari et al. (2013a, Sec.3.2)

Explanation

Linear Least-squares approximation in two-dimension (z = a + b*x + c*y), when x, y and z data have errors, and allowing for intrinsic scatter in the relation.

Outliers are iteratively clipped using the extremely robust FAST-LTS technique by Rousseeuw & van Driessen (2006) See also Rousseeuw (1987)

Calling Sequence

p = lts_planefit(x, y, z, sigx, sigy, sigz, clip=2.6, epsz=True,
                frac=None, pivotx=None, pivoty=None, plot=True, text=True)

The output values are stored as attributes of “p”. See usage example at the bottom of this file.

Input Parameters

x, y, z:

vectors of size N with the measured values.

sigx, sigy, sigz:

vectors of size N with the 1sigma errors in x , y and z.

clip:

values deviating more than clip*sigma from the best fit are considered outliers and are excluded from the plane fit.

epsz:

if True, the intrinsic scatter is printed on the plot.

frac:

fractions of values to include in the LTS stage. Up to a fraction “frac” of the values can be outliers. One must have 0.5 <= frac <= 1 (default frac=0.5).

NOTE: Set frac=1, to turn off outliers detection.

pivotx, pivoty:

if these are not None, then lts_planefit fits the plane:

z = a + b*(x - pivotx) + c*(y - pivoty)

pivotx, pivoty are called x_0, y_0 in eq.(7) of Cappellari et al. (2013) Use of these keywords is strongly recommended, and suggested values are pivotx=np.median(x), pivoty=np.median(y). This keyword is important to reduce the covariance between a, b and c.

plot:

if True a plot of the fit is produced.

text:

if True, the best fitting parameters are printed on the plot.

Output Parameters

The output values are stored as attributed of the lts_linefit class.

p.abc:

best fitting parameters [a, b, c]

p.abc_err:

1*sigma formal errors [a_err, b_err, c_err] on a, b and c.

p.mask:

boolean vector with the same size of x, y and z. It contains True for the elements of (x, y, z) which were included in the fit and False for the outliers which were automatically clipped.

p.rms:

rms = np.std(fit - z) beteween the data and the fitted relation.

p.sig_int:

intrinsic scatter in the z direction around the plane. sig_int is called epsilon_z in eq.(7) of Cappellari et al. (2013).

p.sig_int_err:

1*sigma formal error on sig_int.

---

License

Other/Proprietary License

Copyright (c) 2012-2022 Michele Cappellari

This software is provided as is without any warranty whatsoever. Permission to use, for non-commercial purposes is granted. Permission to modify for personal or internal use is granted, provided this copyright and disclaimer are included in all copies of the software. All other rights are reserved. In particular, redistribution of the code is not allowed.

Changelog

V2.0.20: MC, Oxford, 3 October 2022

• Fixed program stop due to Matplotlib change. Thanks to Hitesh Lala (Heidelberg) for the report.

• Extract documentation from docstrings and show it on PyPi.

V2.0.19: MC, Oxford, 22 January 2021

• Fixed incorrect plot ranges due to a Matplotlib change. Thanks to Davide Bevacqua (unibo.it) for the report.

V2.0.18: MC, Oxford, 17 February 2020

• Properly print significant trailing zeros in results.

V2.0.17: MC, Oxford, 22 January 2020

• Formatted documentation as docstring.

• Included p.rms output.

V2.0.16: MC, Oxford, 27 September 2018

• Fixed clock DeprecationWarning in Python 3.7.

V2.0.15: MC, Oxford, 12 May 2018

• Dropped Python 2.7 support.

V2.0.14: MC, Oxford, 13 April 2018

• Fixed FutureWarning in Numpy 1.14.

V2.0.13: Michele Cappellari, Oxford, 26 July 2017

• Increased upper limit of intrinsic scatter accounting for uncertainty of standard deviation with small samples.

V2.0.12: MC, Oxford, 5 September 2016

• Fixed: store ab errors in p.ab_err as documented. Thanks to Alison Crocker for the correction.

V2.0.11: MC, Oxford, 4 July 2016

• Added capsize=0 in plt.errorbar to prevent error bar caps from showing up in PDF.

V2.0.10: MC, Oxford, 23 January 2016

• Check for non finite values in input.

V2.0.9: MC, Oxford, 8 January 2016

• Use LimeGreen for outliers.

V2.0.8: MC, Oxford, 9 December 2015

• Fixed potential program stop without outliers in Matplotlib 1.5.

• Increased maximum intrinsic scatter for brentq, to avoid possible stops in extreme situations.

V2.0.7: MC, Oxford, 1 October 2015

• Fixed potential program stop without outliers.

V2.0.6: MC, Oxford, 5 September 2015

• Optionally pass a legend label.

V2.0.5: MC, Oxford, 6 July 2015

• Fixed potential program stop without outliers. Thanks to Masato Onodera for a clear report of the problem.

• Output boolean mask instead of good/bad indices.

• Removed lts_linefit_example from this file.

• Print verbose output during calculation.

V2.0.4: MC, Baltimore, 9 June 2015

• Updated documentation.

V2.0.3: MC, Oxford, 10 December 2014

• Uses np.std rather than biweight to estimate scatter upper limit.

V2.0.2: MC, 6 November 2014

• Included _linefit function to avoid np.polyfit bug with weights.

V2.0.1: MC, Oxford, 23 October 2014

• Fixed program stop with zero scatter.

V2.0.0: MC, Portsmouth, 23 June 2014

• Converted from IDL into Python.

V1.0.6: MC, Baltimore, 8 June 2014

• Check that all input vectors have the same size.

V1.0.5: MC, Oxford, 19 September 2013

• Scale line spacing with character size in text output.

V1.0.4: MC, Turku, 10 July 2013

• Fixed program stop affecting earlier versions of IDL. Thanks to Xue-Guang Zhang for reporting the problem and the solution.

V1.0.3: MC, Oxford, 13 March 2013

• Added CLIP keyword.

V1.0.2: MC, Oxford, 1 August 2011

• Added PIVOT keyword.

V1.0.1: MC, Oxford, 28 July 2011

• Included _EXTRA and OVEPLOT, keywords.

V1.0.0: Michele Cappellari, Oxford, 21 March 2011

• Created and tested.