Sensitivity analysis with cross-validation for feature selection and manifold learning

Authors:
Cuixian Chen;Yishi Wang;Yaw Chang;Karl Ricanek
Affiliations:
University of North Carolina Wilmington;University of North Carolina Wilmington;University of North Carolina Wilmington;University of North Carolina Wilmington
Venue:
ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part I
Year:
2012

Citing 7
Cited 0

The nature of statistical learning theory

The nature of statistical learning theory
Approximate statistical tests for comparing supervised classification learning algorithms

Neural Computation
On Comparing Classifiers: Pitfalls toAvoid and a Recommended Approach

Data Mining and Knowledge Discovery
Comparison of synthetic face aging to age progression by forensic sketch artist

VIIP '07 The Seventh IASTED International Conference on Visualization, Imaging and Image Processing
Sensitivity Analysis of k-Fold Cross Validation in Prediction Error Estimation

IEEE Transactions on Pattern Analysis and Machine Intelligence
Generalized multi-ethnic face age-estimation

BTAS'09 Proceedings of the 3rd IEEE international conference on Biometrics: Theory, applications and systems
LIBSVM: A library for support vector machines

ACM Transactions on Intelligent Systems and Technology (TIST)

Quantified Score

Hi-index	0.00

Visualization

Abstract

The performance of a learning algorithm is usually measured in terms of prediction error. It is important to choose an appropriate estimator of the prediction error. This paper analyzes the statistical properties of the K-fold cross-validation prediction error estimator. It investigates how to compare two algorithms statistically. It also analyzes the sensitivity to the changes in the training/test set. Our main contribution is to experimentally study the statistical property of repeated cross-validation to stabilize the prediction error estimation, and thus to reduce the variance of the prediction error estimator. Our simulation results provide an empirical evidence to this conclusion. The experimental study has been performed on PAL dataset for age estimation task.