An introduction to support Vector Machines: and other kernel-based learning methods
An introduction to support Vector Machines: and other kernel-based learning methods
SVMTorch: support vector machines for large-scale regression problems
The Journal of Machine Learning Research
Kernel partial least squares regression in reproducing kernel hilbert space
The Journal of Machine Learning Research
Some greedy learning algorithms for sparse regression and classification with mercer kernels
The Journal of Machine Learning Research
Overview and recent advances in partial least squares
SLSFS'05 Proceedings of the 2005 international conference on Subspace, Latent Structure and Feature Selection
Hi-index | 0.00 |
Kernel partial least squares (KPLS) has been known as a generic kernel regression method and proven to be competitive with other kernel regression methods such as support vector machines for regression (SVM) and kernel ridge regression. Kernel boosted latent features (KBLF) is a variant of KPLS for any differentiable convex loss functions. It provides a more flexible framework for various predictive modeling tasks such as classification with logistic loss and robust regression with L1 norm loss, etc. However, KPLS and KBLF solutions are dense and thus not suitable for large-scale computations. Sparsification of KPLS solutions has been studied for dual and primal forms. For dual sparsity, it requires solving a nonlinear optimization problem at every iteration step and its computational burden limits its applicability to general regression tasks.In this paper, we propose simple heuristics to approximate sparse solutions for KPLS and the framework is also applied for sparsifying KBLF solutions. The algorithm provides an interesting "path" from a maximum residual criterion based algorithm with orthogonality conditions to the dense KPLS/KBLF. With the orthogonality, it differentiates itself from many existing forward selection-type algorithms. The computational advantage is illustrated by benchmark datasets and comparison to SVM is done.