Topic detection and organization of mobile text messages

  • Authors:
  • Ye Tian;Wendong Wang;Xueli Wang;Jinghai Rao;Canfeng Chen;Jian Ma

  • Affiliations:
  • Beijing University of Posts and Telecommunucations, Beijing, China;Beijing University of Posts and Telecommunucations, Beijing, China;Beijing University of Posts and Telecommunucations, Beijing, China;Nokia Research Center, Beijing, China;Nokia Research Center, Beijing, China;Nokia Research Center, Beijing, China

  • Venue:
  • CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

How to organize and visualize big amount of text messages stored on one's mobile phone is a challenging problem, since they can hardly be organized by threads as we do for emails due to lack of necessary metadata such as "subject" and "reply-to". In this paper, we propose an innovative approach based on clustering algorithms and natural language processing methods. We first cluster the text messages into candidate conversations based on their temporal attributes, and then do further analysis using a semantic model based on Latent Dirichlet Allocation (LDA). Considering that the text messages are usually short and sparse, we trained the model using a large scale external data collected from twitter-like web sites, and applied the model to text messages. In the end, the text messages are organized as conversations based on their topics. We evaluated our approach based on 122,359 text messages collected from 50 university students during 6 months.