The multiple imputation quantitative noise corrector

  • Authors:
  • Taghi M. Khoshgoftaar;Jason Van Hulse;Chris Seiffert;Lili Zhao

  • Affiliations:
  • (Correspd. Empirical Software Eng. Lab., Dept. of Comp. Sci. and Eng., Florida Atlantic Univ., Boca Raton, FL 33431, USA. Tel.: +1 561 297 3994/ Fax: +1 561 297 2800/ E-mail: taghi@cse.fau.edu) Fl ...;Florida Atlantic University, Boca Raton, FL, USA;Florida Atlantic University, Boca Raton, FL, USA;Florida Atlantic University, Boca Raton, FL, USA

  • Venue:
  • Intelligent Data Analysis
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

Relatively little attention has been given in the data mining literature to noise handling procedures that deal specifically with a continuous dependent variable. We present a novel procedure that addresses the problem of detecting and correcting noise when the outcome variable is continuous. Our technique uses a procedure for handling missing data called multiple imputation, a well-known statistical methodology based on sound theoretical principles. We demonstrate the utility of our procedure using a real-world dataset with inherent noise and multiple levels of injected noise in numerous carefully designed controlled experiments. Further, we present a comparison with noise correctors developed using five well-known estimation procedures, providing good coverage of the commonly-used classes of estimation techniques such as linear regression, decision trees and neural networks. The results presented in this work demonstrate conclusively the strong noise detection and correction results of our procedure, which outperforms the five competing noise correctors.