Active learning with sampling by uncertainty and density for data annotations

  • Authors:
  • Jingbo Zhu;Huizhen Wang;Benjamin K. Tsou;Matthew Ma

  • Affiliations:
  • Key Laboratory of Medical Image Computing, Ministry of Education, and the Natural Language Processing Lab, Northeastern University Shenyang, China;Key Laboratory of Medical Image Computing, Ministry of Education, and the Natural Language Processing Lab, Northeastern University Shenyang, China;Language Information Sciences Research Center, City University of Hong Kong, Hong Kong, China;Scientific Works, NJ

  • Venue:
  • IEEE Transactions on Audio, Speech, and Language Processing
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

To solve the knowledge bottleneck problem, active learning has been widely used for its ability to automatically select the most informative unlabeled examples for human annotation. One of the key enabling techniques of active learning is uncertainty sampling, which uses one classifier to identify unlabeled examples with the least confidence. Uncertainty sampling often presents problems when outliers are selected. To solve the outlier problem, this paper presents two techniques, sampling by uncertainty and density (SUD) and density-based re-ranking. Both techniques prefer not only the most informative example in terms of uncertainty criterion, but also the most representative example in terms of density criterion. Experimental results of active learning for word sense disambiguation and text classification tasks using six real-world evaluation data sets demonstrate the effectiveness of the proposed methods.