Earpod: eyes-free menu selection using touch input and reactive audio feedback

Authors:
Shengdong Zhao;Pierre Dragicevic;Mark Chignell;Ravin Balakrishnan;Patrick Baudisch
Affiliations:
University of Toronto, Toronto, ON, Canada;University of Toronto, Toronto, ON, Canada;University of Toronto, Toronto, ON, Canada;University of Toronto, Toronto, ON, Canada;Microsoft Research, Redmond, WA
Venue:
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Year:
2007

Citing 32
Cited 52

An empirical comparison of pie vs. linear menus

CHI '88 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The effects of device technology on the usability of advanced telephone functions

CHI '89 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Skip and scan: cleaning up telephone interface

CHI '92 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
VoiceNotes: a speech interface for a hand-held voice notetaker

CHI '93 Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems
An evaluation of earcons for use in auditory human-computer interfaces

CHI '93 Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems
The design and evaluation of marking menus

The design and evaluation of marking menus
Transforming graphical interfaces into auditory interfaces

CHI '95 Conference Companion on Human Factors in Computing Systems
NewsComm: a hand-held interface for interactive access to structured audio

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
SpeechSkimmer: a system for interactively skimming recorded speech

ACM Transactions on Computer-Human Interaction (TOCHI) - Special issue on speech as data
Dynamic Soundscape: mapping time to space for audio browsing

Proceedings of the ACM SIGCHI Conference on Human factors in computing systems
Squeeze me, hold me, tilt me! An exploration of manipulative user interfaces

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Audio hallway: a virtual acoustic environment for browsing

Proceedings of the 11th annual ACM symposium on User interface software and technology
Using nonspeech sounds to provide navigation cues

ACM Transactions on Computer-Human Interaction (TOCHI)
Programming and enjoying music with your eyes closed

Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments

ACM Transactions on Computer-Human Interaction (TOCHI) - Special issue on human-computer interaction with mobile systems
Using while moving: HCI issues in fieldwork environments

ACM Transactions on Computer-Human Interaction (TOCHI) - Special issue on human-computer interaction with mobile systems
The audio notebook: paper and pen interaction with structured speech

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Gestural and audio metaphors as a means of control for mobile devices

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The Psychology of Menu Selection: Designing Cognitive Control at the Human/Computer Interface

The Psychology of Menu Selection: Designing Cognitive Control at the Human/Computer Interface
Maximising screen-space on mobile computing devices

CHI '99 Extended Abstracts on Human Factors in Computing Systems
Curing the menu blues in touch-tone voice interfaces

CHI '01 Extended Abstracts on Human Factors in Computing Systems
Multimodal 'eyes-free' interaction techniques for wearable devices

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Designing the User Interface: Strategies for Effective Human-Computer Interaction (4th Edition)

Designing the User Interface: Strategies for Effective Human-Computer Interaction (4th Edition)
Impromptu: managing networked audio applications for mobile users

Proceedings of the 2nd international conference on Mobile systems, applications, and services
Simple vs. compound mark hierarchical marking menus

Proceedings of the 17th annual ACM symposium on User interface software and technology
AUDITORY ICONS IN LARGE-SCALE COLLABORATIVE ENVIRONMENTS

ACM SIGCHI Bulletin
Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A role for haptics in mobile interaction: initial design using a handheld tactile display prototype

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The benefits of augmenting telephone voice menu navigation with visual browsing and search

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Searching in audio: the utility of transcripts, dichotic presentation, and time-compression

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Zone and polygon menus: using relative position to increase the breadth of multi-stroke marking menus

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Pen-top feedback for paper-based interfaces

UIST '06 Proceedings of the 19th annual ACM symposium on User interface software and technology

Blindsight: eyes-free access to mobile phones

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces

Proceedings of the 10th international conference on Human computer interaction with mobile devices and services
Slide rule: making mobile touch screens accessible to blind people using multi-touch interaction techniques

Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility
An Audio-Haptic Aesthetic Framework Influenced by Visual Theory

HAID '08 Proceedings of the 3rd international workshop on Haptic and Audio Interaction Design
Designing and evaluating multimodal interaction for mobile contexts

ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
TactiMote: a tactile remote control for navigating in long lists

ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
A gesture-based and eyes-free control method for mobile devices

CHI '09 Extended Abstracts on Human Factors in Computing Systems
Take me home: designing safer in-vehicle navigation devices

CHI '09 Extended Abstracts on Human Factors in Computing Systems
Motion marking menus: An eyes-free approach to motion input for handheld devices

International Journal of Human-Computer Studies
Quinze ans de recherche sur les menus: critères et propriétés des techniques de menus

IHM '07 Proceedings of the 19th International Conference of the Association Francophone d'Interaction Homme-Machine
Communicating through gestures without visual feedback

Proceedings of the 2nd International Conference on PErvasive Technologies Related to Assistive Environments
pieTouch: a direct touch gesture interface for interacting with in-vehicle information systems

Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services
Hands on music: physical approach to interaction with digital music

Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services
overView: physically-based vibrotactile feedback for temporal information browsing

Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services
Wavelet menus: a stacking metaphor for adapting marking menus to mobile devices

Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services
Disappearing mobile devices

Proceedings of the 22nd annual ACM symposium on User interface software and technology
SemFeel: a user interface with semantic tactile feedback for mobile touch-screen devices

Proceedings of the 22nd annual ACM symposium on User interface software and technology
Wavelet menu: une adaptation des marking menus pour les dispositifs mobiles

Proceedings of the 21st International Conference on Association Francophone d'Interaction Homme-Machine
Characteristics of pressure-based input for mobile devices

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Why it's quick to be square: modelling new and existing hierarchical menu designs

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Designing eyes-free interaction

HAID'07 Proceedings of the 2nd international conference on Haptic and audio interaction design
Knotty gestures: subtle traces to support interactive use of paper

Proceedings of the International Conference on Advanced Visual Interfaces
Wavelet menus on handheld devices: stacking metaphor for novice mode and eyes-free selection for expert mode

Proceedings of the International Conference on Advanced Visual Interfaces
Timbremap: enabling the visually-impaired to use maps on touch-enabled devices

Proceedings of the 12th international conference on Human computer interaction with mobile devices and services
Designing spatial audio interfaces for mobile devices: supporting multitasking and context information

Proceedings of the 12th international conference on Human computer interaction with mobile devices and services
SqueezeBlock: using virtual springs in mobile devices for eyes-free interaction

UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Sensing foot gestures from the pocket

UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Leveraging proprioception to make mobile phones more accessible to users with visual impairments

Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility
The phone lock: audio and haptic shoulder-surfing resistant PIN entry methods for mobile devices

Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction
Chalk sounds: the effects of dynamic synthesized audio on workspace awareness in distributed groupware

Proceedings of the ACM 2011 conference on Computer supported cooperative work
Visual cues supporting direct touch gesture interaction with in-vehicle information systems

Proceedings of the 2nd International Conference on Automotive User Interfaces and Interactive Vehicular Applications
Air pointing: Design and evaluation of spatial target acquisition with and without visual feedback

International Journal of Human-Computer Studies
Usable gestures for blind people: understanding preference and performance

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Fast or safe?: how performance objectives determine modality output choices while interacting on the move

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A haptic wristwatch for eyes-free interactions

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Two-handed marking menus for multitouch devices

ACM Transactions on Computer-Human Interaction (TOCHI)
Design and evaluation of freehand menu selection interfaces using tilt and pinch gestures

International Journal of Human-Computer Studies
A tempo-sensitive music search engine with multimodal inputs

MIRUM '11 Proceedings of the 1st international ACM workshop on Music information retrieval with user-centered and multimodal strategies
Understanding performance in touch selections: Tap, drag and radial pointing drag with finger, stylus and mouse

International Journal of Human-Computer Studies
Exploring user motivations for eyes-free interaction on mobile devices

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
“Spindex” (Speech Index) Enhances Menus on Touch Screen Devices with Tapping, Wheeling, and Flicking

ACM Transactions on Computer-Human Interaction (TOCHI)
Central executive functions likely mediate the impact of device operation when driving

Proceedings of the 3rd International Conference on Automotive User Interfaces and Interactive Vehicular Applications
LUI: lip in multimodal mobile GUI interaction

Proceedings of the 14th ACM international conference on Multimodal interaction
Designing an effective vibration-based notification interface for mobile phones

Proceedings of the 2013 conference on Computer supported cooperative work
Eyes-free interaction with free-hand gestures and auditory menus

International Journal of Human-Computer Studies
Understanding palm-based imaginary interfaces: the role of visual and tactile cues when browsing

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Ad-binning: leveraging around device space for storing, browsing and retrieving mobile device content

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Watchit: simple gestures and eyes-free interaction for wristwatches and bracelets

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Examining the need for visual feedback during gesture interaction on mobile touchscreen devices for kids

Proceedings of the 12th International Conference on Interaction Design and Children
A study of unidirectional swipe gestures on in-vehicle touch screens

Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications
Comparing three novel multimodal touch interfaces for infotainment menus

Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications
Disambiguation of imprecise input with one-dimensional rotational text entry

ACM Transactions on Computer-Human Interaction (TOCHI)

Quantified Score

Hi-index	0.01

Visualization

Abstract

We present the design and evaluation of earPod: an eyes-free menu technique using touch input and reactive auditory feedback. Studies comparing earPod with an iPod-like visual menu technique on reasonably-sized static menus indicate that they are comparable in accuracy. In terms of efficiency (speed), earPod is initially slower, but outperforms the visual technique within 30 minutes of practice. Our results indicate that earPod is potentially a reasonable eyes-free menu technique for general use, and is a particularly exciting technique for use in mobile device interfaces.