Quality and relevance of domain-specific search: A case study in mental health

  • Authors:
  • Thanh Tin Tang;Nick Craswell;David Hawking;Kathy Griffiths;Helen Christensen

  • Affiliations:
  • Department of Computer Science, ANU Canberra, Australia ACT 0200;CSIRO ICT Centre, ANU Canberra, Australia;CSIRO ICT Centre, ANU Canberra, Australia;Centre for Mental Health Research, ANU Canberra, Australia ACT 0200;Centre for Mental Health Research, ANU Canberra, Australia ACT 0200

  • Venue:
  • Information Retrieval
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

When searching for health information, results quality can be judged against available scientific evidence: Do search engines return advice consistent with evidence based medicine? We compared the performance of domain-specific health and depression search engines against a general-purpose engine (Google) on both relevance of results and quality of advice. Over 101 queries, to which the term `depression' was added if not already present, Google returned more relevant results than those of the domain-specific engines. However, over the 50 treatment-related queries, Google returned 70 pages recommending for or against a well studied treatment, of which 19 strongly disagreed with the scientific evidence. A domain-specific index of 4 sites selected by domain experts was only wrong in 5 of 50 recommendations. Analysis suggests a tension between relevance and quality. Indexing more pages can give a greater number of relevant results, but selective inclusion can give better quality.