On the use of cross-validation to assess performance in multivariate prediction

Authors:
P. Jonathan;W. J. Krzanowski;W. V. McCarthy
Affiliations:
Shell Research Ltd., Thornton, Chester CH1 3SH, UK;School of Mathematical Sciences, University of Exeter, Laver Building, North Park Rd., Exeter EX4 4QE, UK;Shell Research Ltd., Thornton, Chester CH1 3SH, UK
Venue:
Statistics and Computing
Year:
2000

Citing 2
Cited 4

Selection of variables, and assessment of their performance, in mixed-variable discriminant analysis

Computational Statistics & Data Analysis
Introduction to Linear Regression Analysis, Solutions Manual (Wiley Series in Probability and Statistics)

Introduction to Linear Regression Analysis, Solutions Manual (Wiley Series in Probability and Statistics)

Nonlinear non-negative component analysis algorithms

IEEE Transactions on Image Processing
Modeling and prediction of peptide drift times in ion mobility spectrometry using sequence-based and structure-based approaches

Computers in Biology and Medicine
Prediction of radical hysterectomy complications for cervical cancer using computational intelligence methods

ICAISC'12 Proceedings of the 11th international conference on Artificial Intelligence and Soft Computing - Volume Part II
Exploiting inter-sequence correlations for program behavior prediction

Proceedings of the ACM international conference on Object oriented programming systems languages and applications

Quantified Score

Hi-index	0.00

Visualization

Abstract

We describe a Monte Carlo investigation of a number of variants of cross-validation for the assessment of performance of predictive models, including different values of k in leave-k-out cross-validation, and implementation either in a one-deep or a two-deep fashion. We assume an underlying linear model that is being fitted using either ridge regression or partial least squares, and vary a number of design factors such as sample size n relative to number of variables p, and error variance. The investigation encompasses both the non-singular (i.e. n p) and the singular (i.e. n ≤ p) cases. The latter is now common in areas such as chemometrics but has as yet received little rigorous investigation. Results of the experiments enable us to reach some definite conclusions and to make some practical recommendations.