Local overfitting control via leverages

Authors:
Gaétan Monari;Gérard Dreyfus
Affiliations:
Ecole Supérieure de Physique et de Chimie Industrielles de la Ville de Paris, Laboratoire d'Electronique, F 75005 Paris, France and Usinor, DSI/DISA Sollac, F 13776 Fos-sur-Mer Cedex, France;Ecole Supérieure de Physique et de Chimie Industrielles de la Ville de Paris, Laboratoire d'Electronique, F15005, Paris, France
Venue:
Neural Computation
Year:
2002

Citing 7
Cited 14

Numerical recipes in C: the art of scientific computing

Numerical recipes in C: the art of scientific computing
Neural networks and the bias/variance dilemma

Neural Computation
A practical Bayesian framework for backpropagation networks

Neural Computation
Algorithmic stability and sanity-check bounds for leave-one-out cross-validation

COLT '97 Proceedings of the tenth annual conference on Computational learning theory
A systematic and effective supervised learning mechanism based on Jacobian rank deficiency

Neural Computation
Model selection in neural networks

Neural Networks
Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics)

Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics)

Ranking a random feature for variable and feature selection

The Journal of Machine Learning Research
Jacobian Conditioning Analysis for Model Validation

Neural Computation
Reply to the Comments on "Local Overfitting Control via Leverages" in "Jacobian Conditioning Analysis for Model Validation" by I. Rivals and L. Personnaz

Neural Computation
A SOM-based data mining strategy for adaptive modelling of an offset lithographic printing process

Engineering Applications of Artificial Intelligence
A 24-h forecast of ozone peaks and exceedance levels using neural classifiers and weather predictions

Environmental Modelling & Software
An orthogonal forward regression technique for sparse kernel density estimation

Neurocomputing
Ink flow control by multiple models in an offset lithographic printing process

Computers and Industrial Engineering
Orthogonal-least-squares regression: A unified approach for data modelling

Neurocomputing
A hybrid approach to outlier detection in the offset lithographic printing process

Engineering Applications of Artificial Intelligence
Producing pattern examples from "mental" images

Neurocomputing
Probability density estimation with tunable kernels using orthogonal forward regression

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics - Special issue on gait analysis
Graph machines and their applications to computer-aided drug design: a new approach to learning from structured data

UC'06 Proceedings of the 5th international conference on Unconventional Computation
Selecting variables for neural network committees

ISNN'06 Proceedings of the Third international conference on Advances in Neural Networks - Volume Part I
A method for detection of Alzheimer's disease using ICA-enhanced EEG measurements

Artificial Intelligence in Medicine

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a novel approach to dealing with overfitting in black box models. It is based on the leverages of the samples, that is, on the influence that each observation has on the parameters of the model. Since overfitting is the consequence of the model specializing on specific data points during training, we present a selection method for nonlinear models based on the estimation of leverages and confidence intervals. It allows both the selection among various models of equivalent complexities corresponding to different minima of the cost function (e.g., neural nets with the same number of hidden units) and the selection among models having different complexities (e.g., neural nets with different numbers of hidden units). A complete model selection methodology is derived.