Classifying news stories using memory based reasoning

  • Authors:
  • Brij Masand;Gordon Linoff;David Waltz

  • Affiliations:
  • Thinking Machines Corporation, 245 First Street, Cambridge, Massachusetts;Thinking Machines Corporation, 245 First Street, Cambridge, Massachusetts;Thinking Machines Corporation, 245 First Street, Cambridge, Massachusetts and Center for Complex Systems at Brandeis University, Waltham, MA

  • Venue:
  • SIGIR '92 Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval
  • Year:
  • 1992

Quantified Score

Hi-index 0.01

Visualization

Abstract

We describe a method for classifying news stories using Memory Based Reasoning (MBR) a k-nearest neighbor method), that does not require manual topic definitions. Using an already coded training database of about 50,000 stories from the Dow Jones Press Release News Wire, and SEEKER [Stanfill] (a text retrieval system that supports relevance feedback) as the underlying match engine, codes are assigned to new, unseen stories with a recall of about 80% and precision of about 70%. There are about 350 different codes to be assigned. Using a massively parallel supercomputer, we leverage the information already contained in the thousands of coded stories and are able to code a story in about 2 seconds. Given SEEKER, the text retrieval system, we achieved these results in about two person-months. We believe this approach is effective in reducing the development time to implement classification systems involving large number of topics for the purpose of classification, message routing etc.