Human versus machine in the topic distillation task

  • Authors:
  • Mingfang Wu;Gheorghe Muresan;Alistair McLean;Muh-Chyun (Morris) Tang;Ross Wilkinson;Yuelin Li;Hyuk-Jin Lee;Nichloas J. Belkin

  • Affiliations:
  • CSIRO, ICT Center, Melbourne, Australia;Rutgers University, New Brunswick, NJ;CSIRO, ICT Center, Melbourne, Australia;Rutgers University, New Brunswick, NJ;CSIRO, ICT Center, Melbourne, Australia;Rutgers University, New Brunswick, NJ;Rutgers University, New Brunswick, NJ;Rutgers University, New Brunswick, NJ

  • Venue:
  • Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper reports on and discusses a set of user experiments using the TREC 2003 Web interactive track protocol. The focus is on comparing humans and machine algorithms in terms of performance in a topic distillation task. We also investigated the effect of the search results layout in supporting the users' effort.We have demonstrated that machines can perform nearly as well as people on the topic distillation task. Given a system tailored to the task there is significant performance improvement and finally, given a presentation that supports the task, there is strong user satisfaction.