Overlap versus imbalance

  • Authors:
  • Misha Denil;Thomas Trappenberg

  • Affiliations:
  • Faculty of Computer Science, Dalhousie University, Halifax, NS, Canada;Faculty of Computer Science, Dalhousie University, Halifax, NS, Canada

  • Venue:
  • AI'10 Proceedings of the 23rd Canadian conference on Advances in Artificial Intelligence
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper we give a systematic analysis of the relationship between imbalance and overlap as factors influencing classifier performance We demonstrate that these two factors have interdependent effects and that we cannot form a full understanding of their effects by considering them only in isolation Although the imbalance problem can be considered a symptom of the small disjuncts problem which is solved by using larger training sets, the overlap problem is of a fundamentally different character and the performance of learned classifiers can actually be made worse by using more training data when overlap is present We also examine the effects of overlap and imbalance on the complexity of the learned model and demonstrate that overlap is a far more serious factor than imbalance in this respect.