chola-coreloss

A smooth, tunable robust-regression loss function, founded by Gautam Ramesh. Built from two pieces chosen so both have gradients that saturate (bound) for large residuals -- the actual property that gives outlier robustness, not just "grows slower than x^2":

L(r) = ln(r^2 + 1) + huberized_power(r; p, delta)

• ln(r^2 + 1) -- a Cauchy/Lorentzian-style term. Its gradient fades toward zero for large residuals.
• huberized_power(r; p, delta) -- behaves like |r|^p near zero (curvature controlled by p) but continues as a straight line past delta, capping its gradient at a constant -- the same mechanism Huber loss uses, generalized to any p.

Install

pip install chola-coreloss

Usage

import numpy as np
from chola_coreloss import chola_coreloss_grad, chola_coreloss_adaptive_grad

r = pred - target  # residuals

# fixed delta (tune per-dataset via a validation set, like Huber's delta)
grad = chola_coreloss_grad(r, p=2.2, delta=0.7)

# recommended: adaptive delta, re-estimated from the residuals' own
# robust spread (MAD) -- no per-dataset retuning needed
grad = chola_coreloss_adaptive_grad(r, p=2.2, c=1.0)

Why adaptive delta

A fixed delta doesn't transfer between datasets -- it needs to track the scale of your model's actual residuals, which differs per problem. chola_coreloss_adaptive_grad re-estimates delta periodically from the residuals' own robust spread (Median Absolute Deviation), the same technique used in classical M-estimation / IRLS. Tested with c=1.0 on real data (sklearn's diabetes dataset) and synthetic nonlinear data (Friedman #1), clean and with injected label corruption, with no per-dataset retuning:

• Matched or beat plain MSE on every clean-data run.
• Beat fixed-delta Huber loss by 5-7x (RMSE) on corrupted-label data.

Related work

This loss is in the same family as Barron's "A General and Adaptive Robust Loss Function" (CVPR 2019), which also combines a tunable shape parameter with an adaptive scale. chola-coreloss differs in its specific algebraic form (a sum of a Cauchy-style term and a Huberized power term, rather than one continuous interpolating expression) and in how the scale adapts (classical MAD-based re-estimation rather than a learned parameter). If you're citing or building on this work, read Barron's paper too.

License

MIT