Looking at you: fused gyro and face tracking for viewing large imagery on mobile devices

Authors:
Neel Joshi;Abhishek Kar;Michael Cohen
Affiliations:
Microsoft Research, Redmond, Washington, United States;IIT Kanpur, Kanpur, Uttar Pradesh, India & Microsoft Research, Redmond, Washington, United States;Microsoft Research, Redmond, Washington, United States
Venue:
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Year:
2012

Citing 17
Cited 1

Speed-dependent automatic zooming for browsing large documents

UIST '00 Proceedings of the 13th annual ACM symposium on User interface software and technology
Quantitative analysis of scrolling techniques

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Boom chameleon: simultaneous capture of 3D viewpoint, voice and gesture annotations on a spatially-aware display

ACM SIGGRAPH 2003 Papers
Robust Real-Time Face Detection

International Journal of Computer Vision
The radial scroll tool: scrolling support for stylus- or touch-based document navigation

Proceedings of the 17th annual ACM symposium on User interface software and technology
Use your head: exploring face tracking for mobile interaction

CHI '06 Extended Abstracts on Human Factors in Computing Systems
Content-aware scrolling

UIST '06 Proceedings of the 19th annual ACM symposium on User interface software and technology
RubberEdge: reducing clutching by combining position and rate control with elastic feedback

Proceedings of the 20th annual ACM symposium on User interface software and technology
Gaze-enhanced scrolling techniques

Proceedings of the 20th annual ACM symposium on User interface software and technology
Movement-based interaction in camera spaces: a conceptual framework

Personal and Ubiquitous Computing
Rubbing and tapping for precise and rapid selection on touch-screen displays

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
TapTap and MagStick: improving one-handed target acquisition on small touch-screens

AVI '08 Proceedings of the working conference on Advanced visual interfaces
Face Tracking for Spatially Aware Mobile User Interfaces

ICISP '08 Proceedings of the 3rd international conference on Image and Signal Processing
MicroRolls: expanding touch-screen input vocabulary by distinguishing rolls vs. slides of the thumb

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Parallax photography: creating 3D cinematic effects from stills

Proceedings of Graphics Interface 2009
Street slide: browsing street level imagery

ACM SIGGRAPH 2010 papers
PEYE: toward a visual motion based perceptual interface for mobile devices

HCI'07 Proceedings of the 2007 IEEE international conference on Human-computer interaction

Video collections in panoramic contexts

Proceedings of the 26th annual ACM symposium on User interface software and technology

Quantified Score

Hi-index	0.01

Visualization

Abstract

We present a touch-free interface for viewing large imagery on mobile devices. In particular, we focus on viewing paradigms for 360 degree panoramas, parallax image sequences, and long multi-perspective panoramas. We describe a sensor fusion methodology that combines face tracking using a front-facing camera with gyroscope data to produce a robust signal that defines the viewer's 3D position relative to the display. The gyroscopic data provides both low-latency feedback and allows extrapolation of the face position beyond the the field-of-view of the front-facing camera. We also demonstrate a hybrid position and rate control that uses the viewer's 3D position to drive exploration of very large image spaces. We report on the efficacy of the hybrid control vs. position only control through a user study.