Exploratory analysis of highly heterogeneous document collections

  • Authors:
  • Arun S. Maiya;John P. Thompson;Francisco Loaiza-Lemos;Robert M. Rolfe

  • Affiliations:
  • Institute for Defense Analyses, Alexandria, VA, USA;Institute for Defense Analyses, Alexandria, VA, USA;Institute for Defense Analyses, Alexandria, VA, USA;Institute for Defense Analyses, Alexandria, VA, USA

  • Venue:
  • Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present an effective multifaceted system for exploratory analysis of highly heterogeneous document collections. Our system is based on intelligently tagging individual documents in a purely automated fashion and exploiting these tags in a powerful faceted browsing framework. Tagging strategies employed include both unsupervised and supervised approaches based on machine learning and natural language processing. As one of our key tagging strategies, we introduce the KERA algorithm (Keyword Extraction for Reports and Articles). KERA extracts topic-representative terms from individual documents in a purely unsupervised fashion and is revealed to be significantly more effective than state-of-the-art methods. Finally, we evaluate our system in its ability to help users locate documents pertaining to military critical technologies buried deep in a large heterogeneous sea of information.