Dynamic Weighted Majority: A New Ensemble Method for Tracking Concept Drift

  • Authors:
  • Jeremy Z. Kolter;Marcus A. Maloof

  • Affiliations:
  • -;-

  • Venue:
  • ICDM '03 Proceedings of the Third IEEE International Conference on Data Mining
  • Year:
  • 2003

Quantified Score

Hi-index 0.02

Visualization

Abstract

Algorithms for tracking concept drift are important formany applications. We present a general method basedon the Weighted Majority algorithm for using any on-linelearner for concept drift. Dynamic Weighted Majority(DWM) maintains an ensemble of base learners, predictsusing a weighted-majority vote of these "experts",and dynamically creates and deletes experts in response tochanges in performance. We empirically evaluated two experimentalsystems based on the method using incrementalnaive Bayes and Incremental Tree Inducer (ITI) as experts.For the sake of comparison, we also included Blum's implementationof Weighted Majority. On the STAGGER Conceptsand on the SEA Concepts, results suggest that the ensemblemethod learns drifting concepts almost as well as the basealgorithms learn each concept individually. Indeed, we reportthe best overall results for these problems to date.