An introduction to support Vector Machines: and other kernel-based learning methods
An introduction to support Vector Machines: and other kernel-based learning methods
Everything old is new again: a fresh look at historical approaches in machine learning
Everything old is new again: a fresh look at historical approaches in machine learning
| Hi-index | 0.00 |
In this article, for regression problems, we proposed shrinkage methods in training a machine in terms of a regularized cost function. The machine considered in this article is represented by a linear combination of fixed basis functions, in which the number of basis functions, or equivalently, the number of adjustable weights is identical to the number of training data. This setting can be viewed as a nonparametric regression method in statistics. In the regularized cost function employed in this article, the error function is defined by the sum of squared errors and the regularization term is defined by the quadratic form of the weight vector. By assuming i.i.d. Gaussian noise, we proposed three thresholding methods for the orthogonal components which are obtained by eigendecomposition of the Gram matrix of the vectors of basis function outputs. The final weight values are obtained by a linear transformation of the thresholded orthogonal components and are shrinkage estimators. The proposed methods are quite simple and automatic, in which the regularization parameter is enough to be fixed for a small constant value. Simple numerical experiments showed that, by comparing the leave-one-out cross validation method, the computational costs of the proposed methods are strictly low and the generalization capabilities of the trained machines are comparable when the number of data is relatively large.