Evaluation of the Impact of User-Cognitive Styles on the Assessment of Text Summarization

  • Authors:
  • Hien Nguyen;Eugene Santos;Jacob Russell

  • Affiliations:
  • Department of Mathematical and Computer Sciences, University of Wisconsin–Whitewater , Whitewater, WI, USA;Thayer School of Engineering, Dartmouth College, Hanover, NH, USA;Department of Computer Sciences, University of Wisconsin–Milwaukee, Milwaukee, WI, USA

  • Venue:
  • IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Text summarization techniques have been found to be effective with regard to helping users find relevant information faster. The effectiveness and efficiency of a user's performance in an information-seeking task can greatly be improved if he/she needs to only look at a summary that includes the relevant information presented in his/her preferred manner. On the other hand, if the main idea is misrepresented and/or omitted altogether from a summary, it may take users more time to solve a target problem or, even worse, lead users to make incorrect decisions. There is an important need to design a personalized text summarization system that takes into account both what a user is currently interested in and how a user perceives information. The latter factor is referred to as a user's cognitive styles. Although there are some existing approaches that have employed a user's interests to help in the design of a personalized text summarization system, there has been inadequate focus on exploring cognitive styles. This paper aims at studying the impact of a user's cognitive styles when assessing multidocument summaries. In particular, we choose two dimensions of a user's cognitive style—the analytic/wholist and verbal/imagery dimensions—and study their impacts on how a user assesses a summary that was generated from a set of documents. In particular, the type of a document set refers to whether the set's content is loosely or closely related. We use a document set type to explore if there are any differences in the users' assessments of summaries that were generated from sets of different types. The results of this paper show that different users have different assessments with regard to information coverage and the way that information is presented in both loosely and closely related document sets. In addition, we found that the coherency ratings that were given to summaries from the two types of document sets were significantly different between the analytic and wholist groups. This result leads us to investigate the impact of a user's cognitive styles and the following two factors that directly relate to the coherence of a summary: 1) graph entropy and 2) the percentage of stand-alone concepts. We found that these two factors and a user's cognitive styles affect a user's ratings on the coherency of a summary.