On developing robust models for favourability analysis: Model choice, feature sets and imbalanced data

  • Authors:
  • Peter C. R. Lane;Daoud Clarke;Paul Hender

  • Affiliations:
  • School of Computer Science, University of Hertfordshire, College Lane, Hatfield AL10 9AB, Hertfordshire, UK;School of Computer Science, University of Hertfordshire, College Lane, Hatfield AL10 9AB, Hertfordshire, UK and Metrica, Banner Street, London EC1V 9BJ, UK;Metrica, Banner Street, London EC1V 9BJ, UK

  • Venue:
  • Decision Support Systems
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Locating documents carrying positive or negative favourability is an important application within media analysis. This article presents some empirical results on the challenges facing a machine-learning approach to this kind of opinion mining. Some of the challenges include the often considerable imbalance in the distribution of positive and negative samples, changes in the documents over time, and effective training and evaluation procedures for the models. This article presents results on three data sets generated by a media-analysis company, classifying documents in two ways: detecting the presence of favourability, and assessing negative vs. positive favourability. We describe our experiments in developing a machine-learning approach to automate the classification process. We explore the effect of using five different types of features, the robustness of the models when tested on data taken from a later time period, and the effect of balancing the input data by undersampling. We find varying choices for the optimum classifier, feature set and training strategy depending on the task and data set.