A learned saliency predictor for dynamic natural scenes

Authors:
Eleonora Vig;Michael Dorr;Thomas Martinetz;Erhardt Barth
Affiliations:
Institute for Neuro- and Bioinformatics, University of Lübeck, Lübeck, Germany;Institute for Neuro- and Bioinformatics, University of Lübeck, Lübeck, Germany and Schepens Eye Research Institute, Dept. of Ophthalmology, Harvard Medical School, Boston, MA;Institute for Neuro- and Bioinformatics, University of Lübeck, Lübeck, Germany;Institute for Neuro- and Bioinformatics, University of Lübeck, Lübeck, Germany
Venue:
ICANN'10 Proceedings of the 20th international conference on Artificial neural networks: Part III
Year:
2010

Citing 5
Cited 0

A Model of Saliency-Based Visual Attention for Rapid Scene Analysis

IEEE Transactions on Pattern Analysis and Machine Intelligence
Investigation of a sensorimotor system for saccadic scene analysis: an integrated approach

Proceedings of the fifth international conference on simulation of adaptive behavior on From animals to animats 5
Decision-theoretic saliency: Computational principles, biological plausibility, and implications for neurophysiology and psychophysics

Neural Computation
How to find interesting locations in video: a spatiotemporal interest point detector learned from human eye movements

Proceedings of the 29th DAGM conference on Pattern recognition
Foveation scalable video coding with automatic fixation selection

IEEE Transactions on Image Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

We investigate the extent to which eye movements in natural dynamic scenes can be predicted with a simple model of bottom-up saliency, which learns on different visual representations to discriminate between salient and less salient movie regions. Our image representations, the geometrical invariants of the structure tensor, are computed on multiple scales of an anisotropic spatio-temporal multiresolution pyramid. Eye movement data is used to label video locations as salient. For each location, low-dimensional features are extracted on the multiscale representations and used to train a classifier. The quality of the predictor is tested on a large test set of eye movement data and compared with the performance of two state-of-the-art saliency models on this data set. The proposed model demonstrates significant improvement - mean ROC score of 0.665 - over the selected baseline models with ROC scores of 0.625 and 0.635.