Model Stability: A key factor in determining whether an algorithm produces an optimal model from a matching distribution

  • Authors:
  • Kai Ming Ting;Regina Jing Ying Quek

  • Affiliations:
  • -;-

  • Venue:
  • ICDM '03 Proceedings of the Third IEEE International Conference on Data Mining
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper investigates the factors leading to producingsuboptimal models when training and test class distributions(or misclassification costs) are matched. Our resultshows that model stability plays a key role in determiningwhether the algorithm produces an optimal modelfrom a matching distribution (cost). The performance differencebetween a model trained from the matching distribution(cost) and the optimal model generally increases asthe degree of model stability decreases. The practical implicationof our result is that one should only follow theconventional wisdom of using a training class distribution(cost) that matches the test class distribution (cost) to traina classifier if the learning algorithm is known to be stable.