Orientation-aware scene understanding for mobile cameras

Authors:
Jing Wang;Grant Schindler;Irfan Essa
Affiliations:
Georgia Inst. of Technology, Atlanta, Georgia;Georgia Inst. of Technology, Atlanta, Georgia;Georgia Inst. of Technology, Atlanta, Georgia
Venue:
Proceedings of the 2012 ACM Conference on Ubiquitous Computing
Year:
2012

Citing 15
Cited 0

Hybrid Inertial and Vision Tracking for Augmented Reality Registration

VR '99 Proceedings of the IEEE Virtual Reality
Fusion of Vision and Gyro Tracking for Robust Augmented Reality Registration

VR '01 Proceedings of the Virtual Reality 2001 Conference (VR'01)
Context-based vision system for place and object recognition

ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
The Ubiquitous Camera: An In-Depth Study of Camera Phone Use

IEEE Pervasive Computing
Putting Objects in Perspective

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
Recovering Surface Layout from an Image

International Journal of Computer Vision
LabelMe: A Database and Web-Based Tool for Image Annotation

International Journal of Computer Vision
Multi-Class Segmentation with Relative Location Prior

International Journal of Computer Vision
Inertial-aided KLT feature tracking for a moving camera

IROS'09 Proceedings of the 2009 IEEE/RSJ international conference on Intelligent robots and systems
Superparsing: scalable nonparametric image parsing with superpixels

ECCV'10 Proceedings of the 11th European conference on Computer vision: Part V
Gyro-aided feature tracking for a moving camera: fusion, auto-calibration and GPU implementation

International Journal of Robotics Research
TextonBoost: joint appearance, shape and context modeling for multi-class object recognition and segmentation

ECCV'06 Proceedings of the 9th European conference on Computer Vision - Volume Part I
SURF: speeded up robust features

ECCV'06 Proceedings of the 9th European conference on Computer Vision - Volume Part I
Learning and matching multiscale template descriptors for real-time detection, localization and tracking

CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition
Inertial sensor-aligned visual feature descriptors

CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a novel approach that allows anyone to quickly teach their smartphone how to understand the visual world around them. We achieve this visual scene understanding by leveraging a camera-phone's inertial sensors to lead to both a faster and more accurate automatic labeling of the regions of an image into semantic classes (e.g. sky, tree, building). We focus on letting a user train our system from scratch while out in the real world by annotating image regions in situ as training images are captured on a mobile device, making it possible to recognize new environments and new semantic classes on the fly. We show that our approach outperforms existing methods, while at the same time performing data collection, annotation, feature extraction, and image segment classification all on the same mobile device.