LtsFit: Least Trimmed Squares Fitting
Project description
The LtsFit Package
Robust Linear Regression with Scatter in One or Two Dimensions
LtsFit is a Python implementation of the method described in Section 3.2 of Cappellari et al. (2013a) to perform very robust fits of lines or planes to data with errors in all coordinates, while allowing for possible intrinsic scatter. Outliers are iteratively clipped using the robust Least Trimmed Squares (LTS) technique by Rousseeuw & van Driessen (2006).
Attribution
If you use this software for your research, please also cite Cappellari et al. (2013a) where the implementation was described. The BibTeX entry for the paper is:
@ARTICLE{Cappellari2013a,
author = {{Cappellari}, M. and {Scott}, N. and {Alatalo}, K. and
{Blitz}, L. and {Bois}, M. and {Bournaud}, F. and {Bureau}, M. and
{Crocker}, A.~F. and {Davies}, R.~L. and {Davis}, T.~A. and {de Zeeuw},
P.~T. and {Duc}, P.-A. and {Emsellem}, E. and {Khochfar}, S. and
{Krajnovi{\'c}}, D. and {Kuntschner}, H. and {McDermid}, R.~M. and
{Morganti}, R. and {Naab}, T. and {Oosterloo}, T. and {Sarzi}, M. and
{Serra}, P. and {Weijmans}, A.-M. and {Young}, L.~M.},
title = "{The ATLAS$^{3D}$ project - XV. Benchmark for early-type
galaxies scaling relations from 260 dynamical models: mass-to-light
ratio, dark matter, Fundamental Plane and Mass Plane}",
journal = {MNRAS},
eprint = {1208.3522},
year = 2013,
volume = 432,
pages = {1709-1741},
doi = {10.1093/mnras/stt562}
}
Installation
install with:
pip install ltsfit
Without writing access to the global site-packages directory, use:
pip install --user ltsfit
Documentation
See ltsfit/examples and the files docstrings. They are copied by pip within the global folder site-packages.
lts_linefit
Purpose
Best straight-line robust fit to data with errors in both coordinates while fitting for the intrinsic scatter. See Cappellari et al. (2013a, Sec.3.2)
Explanation
Linear Least-squares approximation in one-dimension (y = a + b*x), when both x and y data have errors and allowing for intrinsic scatter in the relation.
Outliers are iteratively clipped using the extremely robust FAST-LTS technique by Rousseeuw & van Driessen (2006) See also Rousseeuw (1987)
Calling Sequence
p = lts_linefit(x, y, sigx, sigy, clip=2.6, epsy=True, corr=True,
frac=None, pivot=None, plot=True, text=True)The output values are stored as attributes of “p”. See usage example at the bottom of this file.
Input Parameters
- x, y:
vectors of size N with the measured values.
- sigx, sigy:
vectors of size N with the 1sigma errors in x and y.
- clip:
values deviating more than clip*sigma from the best fit are considered outliers and are excluded from the linear fit.
- epsy:
if True, the intrinsic scatter is printed on the plot.
- corr:
if True, the correlation coefficients are printed on the plot.
- frac:
fractions of values to include in the LTS stage. Up to a fraction “frac” of the values can be outliers. One must have 0.5 < frac < 1 (default frac=0.5).
NOTE: Set frac=1, to turn off outliers detection.
- label:
optional string label to create a legend outside the procedure.
- pivot:
if this is not None, then lts_linefit fits the relation:
y = a + b*(x - pivot)
pivot is called x_0 in eq.(6) of Cappellari et al. (2013) Use of this keyword is strongly recommended and a suggested value is pivot ~ np.mean(x). This keyword is important to reduce the covariance between a and b.
- plot:
if True a plot of the fit is produced.
- text:
if True, the best fitting parameters are printed on the plot.
Output Parameters
The output values are stored as attributed of the lts_linefit class.
- p.ab:
best fitting parameters [a, b]
- p.ab_err:
1*sigma formal errors [a_err, b_err] on a and b.
- p.mask:
boolean vector with the same size of x and y. It contains True for the elements of (x, y) which were included in the fit and False for the outliers which were automatically clipped.
- p.rms:
rms = np.std(fit - y) beteween the data and the fitted relation.
- p.sig_int:
intrinsic scatter around the linear relation. sig_int is called epsilon_y in eq.(6) of Cappellari et al. (2013).
- p.sig_int_err:
1*sigma formal error on sig_int.
lts_planefit
Purpose
Best plane robust fit to data with errors in all three coordinates and fitting for the intrinsic scatter. See Cappellari et al. (2013a, Sec.3.2)
Explanation
Linear Least-squares approximation in two-dimension (z = a + b*x + c*y), when x, y and z data have errors, and allowing for intrinsic scatter in the relation.
Outliers are iteratively clipped using the extremely robust FAST-LTS technique by Rousseeuw & van Driessen (2006) See also Rousseeuw (1987)
Calling Sequence
p = lts_planefit(x, y, z, sigx, sigy, sigz, clip=2.6, epsz=True,
frac=None, pivotx=None, pivoty=None, plot=True, text=True)The output values are stored as attributes of “p”. See usage example at the bottom of this file.
Input Parameters
- x, y, z:
vectors of size N with the measured values.
- sigx, sigy, sigz:
vectors of size N with the 1sigma errors in x , y and z.
- clip:
values deviating more than clip*sigma from the best fit are considered outliers and are excluded from the plane fit.
- epsz:
if True, the intrinsic scatter is printed on the plot.
- frac:
fractions of values to include in the LTS stage. Up to a fraction “frac” of the values can be outliers. One must have 0.5 <= frac <= 1 (default frac=0.5).
NOTE: Set frac=1, to turn off outliers detection.
- pivotx, pivoty:
if these are not None, then lts_planefit fits the plane:
z = a + b*(x - pivotx) + c*(y - pivoty)
pivotx, pivoty are called x_0, y_0 in eq.(7) of Cappellari et al. (2013) Use of these keywords is strongly recommended, and suggested values are pivotx=np.median(x), pivoty=np.median(y). This keyword is important to reduce the covariance between a, b and c.
- plot:
if True a plot of the fit is produced.
- text:
if True, the best fitting parameters are printed on the plot.
Output Parameters
The output values are stored as attributed of the lts_linefit class.
- p.abc:
best fitting parameters [a, b, c]
- p.abc_err:
1*sigma formal errors [a_err, b_err, c_err] on a, b and c.
- p.mask:
boolean vector with the same size of x, y and z. It contains True for the elements of (x, y, z) which were included in the fit and False for the outliers which were automatically clipped.
- p.rms:
rms = np.std(fit - z) beteween the data and the fitted relation.
- p.sig_int:
intrinsic scatter in the z direction around the plane. sig_int is called epsilon_z in eq.(7) of Cappellari et al. (2013).
- p.sig_int_err:
1*sigma formal error on sig_int.
License
Other/Proprietary License
Copyright (c) 2012-2022 Michele Cappellari
This software is provided as is without any warranty whatsoever. Permission to use, for non-commercial purposes is granted. Permission to modify for personal or internal use is granted, provided this copyright and disclaimer are included in all copies of the software. All other rights are reserved. In particular, redistribution of the code is not allowed.
Changelog
- V2.0.20: MC, Oxford, 3 October 2022
Fixed program stop due to Matplotlib change. Thanks to Hitesh Lala (Heidelberg) for the report.
Extract documentation from docstrings and show it on PyPi.
- V2.0.19: MC, Oxford, 22 January 2021
Fixed incorrect plot ranges due to a Matplotlib change. Thanks to Davide Bevacqua (unibo.it) for the report.
- V2.0.18: MC, Oxford, 17 February 2020
Properly print significant trailing zeros in results.
- V2.0.17: MC, Oxford, 22 January 2020
Formatted documentation as docstring.
Included p.rms output.
- V2.0.16: MC, Oxford, 27 September 2018
Fixed clock DeprecationWarning in Python 3.7.
- V2.0.15: MC, Oxford, 12 May 2018
Dropped Python 2.7 support.
- V2.0.14: MC, Oxford, 13 April 2018
Fixed FutureWarning in Numpy 1.14.
- V2.0.13: Michele Cappellari, Oxford, 26 July 2017
Increased upper limit of intrinsic scatter accounting for uncertainty of standard deviation with small samples.
- V2.0.12: MC, Oxford, 5 September 2016
Fixed: store ab errors in p.ab_err as documented. Thanks to Alison Crocker for the correction.
- V2.0.11: MC, Oxford, 4 July 2016
Added capsize=0 in plt.errorbar to prevent error bar caps from showing up in PDF.
- V2.0.10: MC, Oxford, 23 January 2016
Check for non finite values in input.
- V2.0.9: MC, Oxford, 8 January 2016
Use LimeGreen for outliers.
- V2.0.8: MC, Oxford, 9 December 2015
Fixed potential program stop without outliers in Matplotlib 1.5.
Increased maximum intrinsic scatter for brentq, to avoid possible stops in extreme situations.
- V2.0.7: MC, Oxford, 1 October 2015
Fixed potential program stop without outliers.
- V2.0.6: MC, Oxford, 5 September 2015
Optionally pass a legend label.
- V2.0.5: MC, Oxford, 6 July 2015
Fixed potential program stop without outliers. Thanks to Masato Onodera for a clear report of the problem.
Output boolean mask instead of good/bad indices.
Removed lts_linefit_example from this file.
Print verbose output during calculation.
- V2.0.4: MC, Baltimore, 9 June 2015
Updated documentation.
- V2.0.3: MC, Oxford, 10 December 2014
Uses np.std rather than biweight to estimate scatter upper limit.
- V2.0.2: MC, 6 November 2014
Included _linefit function to avoid np.polyfit bug with weights.
- V2.0.1: MC, Oxford, 23 October 2014
Fixed program stop with zero scatter.
- V2.0.0: MC, Portsmouth, 23 June 2014
Converted from IDL into Python.
- V1.0.6: MC, Baltimore, 8 June 2014
Check that all input vectors have the same size.
- V1.0.5: MC, Oxford, 19 September 2013
Scale line spacing with character size in text output.
- V1.0.4: MC, Turku, 10 July 2013
Fixed program stop affecting earlier versions of IDL. Thanks to Xue-Guang Zhang for reporting the problem and the solution.
- V1.0.3: MC, Oxford, 13 March 2013
Added CLIP keyword.
- V1.0.2: MC, Oxford, 1 August 2011
Added PIVOT keyword.
- V1.0.1: MC, Oxford, 28 July 2011
Included _EXTRA and OVEPLOT, keywords.
- V1.0.0: Michele Cappellari, Oxford, 21 March 2011
Created and tested.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
