Preventing Over-Fitting during Model Selection via Bayesian Regularisation of the Hyper-Parameters
The Journal of Machine Learning Research
Mining competent case bases for case-based reasoning
Artificial Intelligence
| Hi-index | 0.00 |
Mika et al. [Fisher discriminant analysis with kernels] introduce a non-linear formulation of Fisher's linear discriminant, based the now familiar "kernel trick", demonstrating state-of-the-art performance on a wide range of real-world benchmark datasets.In this paper, we show that the usual regularisation parameter can be adjusted so as to minimise the leave-one-out cross-validation error with a computational complexity of only O(l^2) operations, where l is the number of training patterns, rather than the O(l^4) operations required for a naïeve implementation of the leave-one-out procedure.This procedure is then used to form a component of an efficient heirarchical model selection strategy where the regularisation parameter is optimised within the inner loop while the kernel parameter are optimised in the outer loop.