Automatic dimensionality selection from the scree plot via the use of profile likelihood

  • Authors:
  • Mu Zhu;Ali Ghodsi

  • Affiliations:
  • Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Ont., Canada N2L 3G1;Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Ont., Canada N2L 3G1

  • Venue:
  • Computational Statistics & Data Analysis
  • Year:
  • 2006

Quantified Score

Hi-index 0.03

Visualization

Abstract

Most dimension reduction techniques produce ordered coordinates so that only the first few coordinates need be considered in subsequent analyses. The choice of how many coordinates to use is often made with a visual heuristic, i.e., by making a scree plot and looking for a ''big gap'' or an ''elbow.'' In this article, we present a simple and automatic procedure to accomplish this goal by maximizing a simple profile likelihood function. We give a wide variety of both simulated and real examples.