InverseProblem 1.0

Iterative Approach to using Tikhonov Regularizaiton for inverting a matrix

InverseProblem

InverseProblem.invert(A, b, k, l)

Function that operates on any sized ill conditioned matrix A and noisy vecotr b, to solve

Ax=b by inverting A.

It uses the Tikhonov Regularization method, but rather that using its analytic solution it

uses its iterative solution, which converges to the noiseless solution for b.

A is any dimension matrix m x n

b is a vector m x 1

x is a vector n x 1

k is the number of iterations required

l is lambda, or the dampenign parameter in the regularizaiton problem

In many inversion problems the data frame or matrix X is ill-conditioned, meaning the matrix iself is close to singular (e.g. a Hilbert matrix that has several close to dependent columns),and the solution, b, is therefore is noisy. A small perturbation in the matrix X will lead to wild adjustements in the solution, b, as is well-known.

One way to deal with such instability is to regularize the problem and solve a “nearby” problem using Lasso methods or the more general Tikhanov reguarlization problem, which add a penalty, lambda, to the optimizaiton problem. However, currently, no efficient method exists for searching for lambda.

In this new innovative method, we have derived an iterative approach to solving the general Tikhanov regularization problem, which converges to the noiseless solution, does not depend strongly on the choice of lambda, and yet still avoids the inversion problem.

This alogirithm is written in a few lines of code using Python’s numpy package, and primarily relies on the SVD composition. The solution is applicable to many fields in image processing and data, and the code, as we’ll see in the talk, is quite simple.