Combined input variable selection and model complexity control for nonlinear regression

Authors:
Timo Similä;Jarkko Tikka
Affiliations:
Laboratory of Computer and Information Science, Helsinki University of Technology, P.O. Box 5400, FIN-02015 HUT, Finland and Xtract Hitsaajankatu 22, FIN-00810 Helsinki, Finland;Laboratory of Computer and Information Science, Helsinki University of Technology, P.O. Box 5400, FIN-02015 HUT, Finland
Venue:
Pattern Recognition Letters
Year:
2009

Citing 17
Cited 3

Multilayer feedforward networks are universal approximators

Neural Networks
Connectionist nonparametric regression: multilayer feedforward networks can learn arbitrary mappings

Neural Networks
Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks

Neural Networks
Optimal brain damage

Advances in neural information processing systems 2
Original Contribution: A scaled conjugate gradient algorithm for fast supervised learning

Neural Networks
Selection of relevant features and examples in machine learning

Artificial Intelligence - Special issue on relevance
Wrappers for feature subset selection

Artificial Intelligence - Special issue on relevance
Model selection in neural networks

Neural Networks
An introduction to variable and feature selection

The Journal of Machine Learning Research
Dimensionality reduction via sparse support vector machines

The Journal of Machine Learning Research
Neural input selection-A fast model-based approach

Neurocomputing
Resampling methods for parameter-free and robust feature selection with mutual information

Neurocomputing
Input selection for radial basis function networks by constrained optimization

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Analysis of fast input selection: application in time series prediction

ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part II
Avoiding Pitfalls in Neural Network Research

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
Integrated feature architecture selection

IEEE Transactions on Neural Networks
Neural-network construction and selection in nonlinear modeling

IEEE Transactions on Neural Networks

Selecting useful features for personal credit risk analysis

International Journal of Business Information Systems
On calibration data selection: The case of stormwater quality regression models

Environmental Modelling & Software
Handwritten characters recognition based on nature-inspired computing and neuro-evolution

Applied Intelligence

Quantified Score

Hi-index	0.10

Visualization

Abstract

Choosing a useful combination of input variables and an appropriate complexity of the model is an essential task in nonlinear regression analysis because of the risk of overfitting. This article provides a workable solution for the multilayer perceptron model. An initial structure of the model, including all the input variables, is fixed in the beginning. Only the most useful input variables and hidden nodes remain effective when the model is fitted with the proposed penalization method. The method is tested on three benchmark data sets. Experimental results show that the removal of useless input variables and hidden nodes from the model improves its generalization capability. In addition, the proposed method compares favorably with respect to other penalization methods.