Can text summaries help predict ratings? a case study of movie reviews

Authors:
Horacio Saggion;Elena Lloret;Manuel Palomar
Affiliations:
Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain;Department of Software and Computing Systems, University of Alicante, Alicante, Spain;Department of Software and Computing Systems, University of Alicante, Alicante, Spain
Venue:
NLDB'12 Proceedings of the 17th international conference on Applications of Natural Language Processing and Information Systems
Year:
2012

Citing 7
Cited 1

Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales

ACL '05 Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics
Automatic summarising: The state of the art

Information Processing and Management: an International Journal
Opinion Mining and Sentiment Analysis

Foundations and Trends in Information Retrieval
Adapting svm for data sparseness and imbalance: A case study in information extraction

Natural Language Engineering
Seeing stars when there aren't many stars: graph-based semi-supervised learning for sentiment categorization

TextGraphs-1 Proceedings of the First Workshop on Graph Based Methods for Natural Language Processing
Experiments on summary-based opinion classification

CAAGET '10 Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text
The bag-of-opinions method for review rating prediction from sparse text patterns

COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics

Editorial: COMPENDIUM: A text summarization system for generating abstracts of research papers

Data & Knowledge Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents an analysis of the rating inference task --- the task of correctly predicting the rating associated to a review, in the context of movie reviews. For achieving this objective, we study the use of automatic text summaries instead of the full reviews. An extrinsic evaluation framework is proposed, where full reviews and different types of summaries (positional, generic and sentiment-based) of several compression rates (from 10% to 50%) are evaluated. We are facing a difficult task; however, the results obtained are very promising and demonstrate that summaries are appropriate for the rating inference problem, performing at least equally to the full reviews when summaries are at least 30% compression rate. Moreover, we also find out that the way the review is organised, as well as the style of writing, strongly determines the performance of the different types of summaries.