Neurofuzzy adaptive modelling and control
Neurofuzzy adaptive modelling and control
Machine Learning
Adaptive modelling, estimation and fusion from data: a neurofuzzy approach
Adaptive modelling, estimation and fusion from data: a neurofuzzy approach
Atomic Decomposition by Basis Pursuit
SIAM Review
Sparse bayesian learning and the relevance vector machine
The Journal of Machine Learning Research
Backward Elimination Methods for Associative Memory Network Pruning
International Journal of Hybrid Intelligent Systems
Regularization in the selection of radial basis function centers
Neural Computation
Sparse modeling using orthogonal forward regression with PRESS statistic and regularization
IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
IEEE Transactions on Fuzzy Systems
IEEE Transactions on Information Theory
IEEE Transactions on Neural Networks
An introduction to kernel-based learning algorithms
IEEE Transactions on Neural Networks
RBF neural network center selection based on Fisher ratio class separability measure
IEEE Transactions on Neural Networks
IEEE Transactions on Neural Networks
Model selection approaches for non-linear system identification: a review
International Journal of Systems Science
Hi-index | 0.00 |
A fundamental principle in practical nonlinear data modeling is the parsimonious principle of constructing the minimal model that explains the training data well. Leave-one-out (LOO) cross validation is often used to estimate generalization errors by choosing amongst different network architectures (M. Stone, "Cross validatory choice and assessment of statistical predictions", J. R. Stast. Soc., Ser. B, 36, pp. 117-147, 1974). Based upon the minimization of LOO criteria of either the mean squares of LOO errors or the LOO misclassification rate respectively, we present two backward elimination algorithms as model post-processing procedures for regression and classification problems. The proposed backward elimination procedures exploit an orthogonalization procedure to enable the orthogonality between the subspace as spanned by the pruned model and the deleted regressor. Subsequently, it is shown that the LOO criteria used in both algorithms can be calculated via some analytic recursive formula, as derived in this contribution, without actually splitting the estimation data set so as to reduce computational expense. Compared to most other model construction methods, the proposed algorithms are advantageous in several aspects; (i) There are no tuning parameters to be optimized through an extra validation data set; (ii) The procedure is fully automatic without an additional stopping criteria; and (iii) The model structure selection is directly based on model generalization performance. The illustrative examples on regression and classification are used to demonstrate that the proposed algorithms are viable post-processing methods to prune a model to gain extra sparsity and improved generalization.