On Tikhonov regularization, bias and variance in nonlinear system identification
Automatica (Journal of IFAC)
Regularization by Truncated Total Least Squares
SIAM Journal on Scientific Computing
Partial least squares regression
Proceedings of the second international workshop on Recent advances in total least squares techniques and errors-in-variables modeling
The symmetric eigenvalue problem
The symmetric eigenvalue problem
LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares
ACM Transactions on Mathematical Software (TOMS)
Brief paper: Geometric properties of partial least squares for process monitoring
Automatica (Journal of IFAC)
| Hi-index | 22.15 |
In this paper it is shown that the Partial Least-Squares (PLS) algorithm for univariate data is equivalent to using a truncated Cayley-Hamilton polynomial expression of degree 1@?a@?r for the matrix inverse (X^TX)^-^1@?R^r^x^r which is used to compute the least-squares (LS) solution. Furthermore, the a coefficients in this polynomial are computed as the optimal LS solution (minimizing parameters) to the prediction error. The resulting solution is non-iterative. The solution can be expressed in terms of a matrix inverse and is given by B"P"L"S=K"a(K"a^TX^TXK"a)^-^1K"a^TX^TY where K"a@?R^r^x^a is the controllability (Krylov) matrix for the pair (X^TX,X^TY). The iterative PLS algorithm for computing the orthogonal weighting matrix W"a as presented in the literature, is shown here to be equivalent to computing an orthonormal basis (using, e.g. the QR algorithm) for the column space of K"a. The PLS solution can equivalently be computed as B"P"L"S=W"a(W"a^TX^TXW"a)^-^1W"a^TX^TY, where W"a is the Q (orthogonal) matrix from the QR decomposition K"a=W"aR. Furthermore, we have presented an optimal and non-iterative truncated Cayley-Hamilton polynomial LS solution for multivariate data. The free parameters in this solution is found as the minimizing solution of a prediction error criterion.