PLS-based recursive feature elimination for high-dimensional small sample

  • Authors:
  • Wenjie You;Zijiang Yang;Guoli Ji

  • Affiliations:
  • Department of Automation, Xiamen University, 361005 Xiamen, China and School of Information Technology, York University, Toronto M3J 1P3, Canada;School of Information Technology, York University, Toronto M3J 1P3, Canada;Department of Automation, Xiamen University, 361005 Xiamen, China and Innovation Center for Cell Biology Research, Xiamen University, 361102 Xiamen, China

  • Venue:
  • Knowledge-Based Systems
  • Year:
  • 2014

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper focused on feature selection for high-dimensional small samples (HDSS). We first presented a general analytical framework for feature selection on a HDSS including selection strategy (single-feature ranking and multi-feature ranking) and evaluation criteria (feature subset consistency and compactness). Then we proposed partial least squares (PLS) based feature selection methods for HDSS and two theorems. The proposed methodologies include a PLS model for classification, parameter selection, PLSRanking, and PLS-based recursive feature elimination. Furthermore, we compared our proposed methods with several existing feature selection methods such as Support Vector Machine (SVM) based feature selection, SVM-based recursive feature elimination (SVMRFE), Random Forest (RF) based feature selection, RF-based recursive feature elimination (RFRFE), ReliefF algorithm and ReliefF-based recursive feature elimination (ReliefFRFE). Using twelve high-dimensional datasets from different areas of research, we evaluated the results in terms of accuracy (sensitivity and specificity), running time, and the feature subset consistency and compactness. The analysis demonstrated that the proposed approach from our research performed very well when handling both two-category and multi-category problems.