Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope

  • Authors:

  • Aude Oliva;Antonio Torralba

  • Affiliations:

  • Harvard Medical School and the Brigham and Women's Hospital, 221 Longwood Ave., Boston, MA 02115. oliva@search.bwh.harvard.edu;Department of Brain and Cognitive Sciences, MIT, 45 Carleton Street, Cambridge, MA 02139. torralba@ai.mit.edu

  • Venue:
  • International Journal of Computer Vision
  • Year:
  • 2001

Quantified Score

Hi-index 0.04

Visualization

Abstract

In this paper, we propose a computational model of the recognition of real world scenes that bypasses the segmentation and the processing of individual objects or regions. The procedure is based on a very low dimensional representation of the scene, that we term the Spatial Envelope. We propose a set of perceptual dimensions (naturalness, openness, roughness, expansion, ruggedness) that represent the dominant spatial structure of a scene. Then, we show that these dimensions may be reliably estimated using spectral and coarsely localized information. The model generates a multidimensional space in which scenes sharing membership in semantic categories (e.g., streets, highways, coasts) are projected closed together. The performance of the spatial envelope model shows that specific information about object shape or identity is not a requirement for scene categorization and that modeling a holistic representation of the scene informs about its probable semantic category.