Synthesizing image representations of linguistic and topological features for predicting areas of attention

Authors:
Pascual Martínez-Gómez;Tadayoshi Hara;Chen Chen;Kyohei Tomita;Yoshinobu Kano;Akiko Aizawa
Affiliations:
National Institute of Informatics, Japan, The University of Tokyo, Japan;National Institute of Informatics, Japan;National Institute of Informatics, Japan, The University of Tokyo, Japan;National Institute of Informatics, Japan, The University of Tokyo, Japan;National Institute of Informatics, Japan,PRESTO, Japan Science and Technology Agency, Japan;National Institute of Informatics, Japan
Venue:
PRICAI'12 Proceedings of the 12th Pacific Rim international conference on Trends in Artificial Intelligence
Year:
2012

Citing 8
Cited 0

WordNet: a lexical database for English

Communications of the ACM
A Model of Saliency-Based Visual Attention for Rapid Scene Analysis

IEEE Transactions on Pattern Analysis and Machine Intelligence
Feature forest models for probabilistic hpsg parsing

Computational Linguistics
Eye tracking as an MT evaluation technique

Machine Translation
A cognitive cost model of annotations based on eye-tracking data

ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
The text 2.0 framework: writing web-based gaze-controlled realtime applications quickly and easily

Proceedings of the 2010 workshop on Eye gaze in intelligent human machine interaction
Image registration for text-gaze alignment

Proceedings of the 2012 ACM international conference on Intelligent User Interfaces
Quantitative analysis and inference on gaze data using natural language processing techniques

Proceedings of the 2012 ACM international conference on Intelligent User Interfaces

Quantified Score

Hi-index	0.00

Visualization

Abstract

Depending on the reading objective or task, text portions with certain linguistic features require more user attention to maximize the level of understanding. The goal is to build a predictor of these text areas. Our strategy consists in synthesizing image representations of linguistic features, that allows us to use natural language processing techniques while preserving the topology of the text. Eye-tracking technology allows us to precisely observe the identity of fixated words on a screen and their fixation duration. Then, we estimate the scaling factors of a linear combination of image representations of linguistic features that best explain certain gaze evidence, which leads us to a quantification of the influence of linguistic features in reading behavior. Finally, we can compute saliency maps that contain a prediction of the most interesting or cognitive demanding areas along the text. We achieve an important prediction accuracy of the text areas that require more attention for users to maximize their understanding in certain reading tasks, suggesting that linguistic features are good signals for prediction.