Multimodal 'eyes-free' interaction techniques for wearable devices

Authors:
Stephen Brewster;Joanna Lumsden;Marek Bell;Malcolm Hall;Stuart Tasker
Affiliations:
University of Glasgow, U.K.;National Research Council of Canada, Fredericton, New Brunswick, Canada;University of Glasgow, U.K.;University of Glasgow, U.K.;University of Glasgow, U.K.
Venue:
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Year:
2003

Citing 11
Cited 68

Multidimensional audio window management

International Journal of Man-Machine Studies - Computer-supported cooperative work and groupware. part 2
3-D sound for virtual reality and multimedia

3-D sound for virtual reality and multimedia
A generic direct-manipulation 3D-auditory environment for hierarchical navigation in non-visual interaction

Assets '96 Proceedings of the second annual ACM conference on Assistive technologies
Head pointing and speech control as a hands-free interface to desktop computing

Assets '98 Proceedings of the third international ACM conference on Assistive technologies
Squeeze me, hold me, tilt me! An exploration of manipulative user interfaces

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Bullseye! when Fitts' law doesn't fit

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Using nonspeech sounds to provide navigation cues

ACM Transactions on Computer-Human Interaction (TOCHI)
Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments

ACM Transactions on Computer-Human Interaction (TOCHI) - Special issue on human-computer interaction with mobile systems
Gestural and audio metaphors as a means of control for mobile devices

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Fundamentals of Wearable Computers and Augumented Reality

Fundamentals of Wearable Computers and Augumented Reality
Overcoming the Lack of Screen Space on Mobile Computers

Personal and Ubiquitous Computing

A paradigm shift: alternative interaction techniques for use with mobile & wearable devices

CASCON '03 Proceedings of the 2003 conference of the Centre for Advanced Studies on Collaborative research
Ubiquitous computing for firefighters: field studies and prototypes of large displays for incident command

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Mixed-mode dialogue information access for the visually impaired

ACM SIGACCESS Accessibility and Computing
Effects of reproduction equipment on interaction with a spatial audio interface

CHI '05 Extended Abstracts on Human Factors in Computing Systems
A comparison of feedback cues for enhancing pointing efficiency in interaction with spatial audio displays

Proceedings of the 7th international conference on Human computer interaction with mobile devices & services
Mobile text entry: relationship between walking speed and text input task difficulty

Proceedings of the 7th international conference on Human computer interaction with mobile devices & services
Headphones with touch control

Proceedings of the 7th international conference on Human computer interaction with mobile devices & services
Ontology-Based User Modeling in an Augmented Audio Reality System for Museums

User Modeling and User-Adapted Interaction
Effects of feedback, mobility and index of difficulty on deictic spatial audio target acquisition in the horizontal plane

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
ec(h)o: situated play in a tangible and audio museum guide

DIS '06 Proceedings of the 6th conference on Designing Interactive systems
Can we do without GUIs? Gesture and speech interaction with a patient information system

Personal and Ubiquitous Computing
Human-centered computing: a multimedia perspective

MULTIMEDIA '06 Proceedings of the 14th annual ACM international conference on Multimedia
Earpod: eyes-free menu selection using touch input and reactive audio feedback

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Multimodal human-computer interaction: A survey

Computer Vision and Image Understanding
Situated play in a tangible interface and adaptive audio museum guide

Personal and Ubiquitous Computing
Spoken words: activating text-to-speech through eye closure

CHI '08 Extended Abstracts on Human Factors in Computing Systems
A user study of auditory versus visual interfaces for use while driving

International Journal of Human-Computer Studies
Experiments in mobile spatial audio-conferencing: key-based and gesture-based interaction

Proceedings of the 10th international conference on Human computer interaction with mobile devices and services
Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces

Proceedings of the 10th international conference on Human computer interaction with mobile devices and services
Visual feedback and different frames of reference: the impact on gesture interaction techniques for wearable computing

Proceedings of the 10th international conference on Human computer interaction with mobile devices and services
Wrist rotation for interaction in mobile contexts

Proceedings of the 10th international conference on Human computer interaction with mobile devices and services
Tlk or txt? Using voice input for SMS composition

Personal and Ubiquitous Computing
Multimodal Interaction: Real Context Studies on Mobile Digital Artefacts

HAID '08 Proceedings of the 3rd international workshop on Haptic and Audio Interaction Design
An Audio-Haptic Interface Concept Based on Depth Information

HAID '08 Proceedings of the 3rd international workshop on Haptic and Audio Interaction Design
An audio-haptic interface based on auditory depth cues

ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
A hand clap interface for sonic interaction with the computer

CHI '09 Extended Abstracts on Human Factors in Computing Systems
A gesture-based and eyes-free control method for mobile devices

CHI '09 Extended Abstracts on Human Factors in Computing Systems
Exploring cues and rhythm for designing multimodal tools to support mobile users in wayfinding

CHI '09 Extended Abstracts on Human Factors in Computing Systems
Motion marking menus: An eyes-free approach to motion input for handheld devices

International Journal of Human-Computer Studies
Bearing-based selection in mobile spatial interaction

Personal and Ubiquitous Computing
Communicating through gestures without visual feedback

Proceedings of the 2nd International Conference on PErvasive Technologies Related to Assistive Environments
Head tilting for interaction in mobile contexts

Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services
Pressure-based text entry for mobile devices

Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services
Exploring Multimodal Navigation Aids for Mobile Users

INTERACT '09 Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part I
From Paper to PDA: Design and Evaluation of a Clinical Ward Instruction on a Mobile Device

INTERACT '09 Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part II
Brief encounters: Sensing, modeling and visualizing urban mobility and copresence networks

ACM Transactions on Computer-Human Interaction (TOCHI)
A simple index for multimodal flexibility

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Heads-up engagement with the real world: multimodal techniques for bridging the physical-digital divide

CHI '10 Extended Abstracts on Human Factors in Computing Systems
“Writing with music”: Exploring the use of auditory feedback in gesture interfaces

ACM Transactions on Applied Perception (TAP)
Designing eyes-free interaction

HAID'07 Proceedings of the 2nd international conference on Haptic and audio interaction design
mCube: towards a versatile gesture input device for ubiquitous computing environments

UCS'07 Proceedings of the 4th international conference on Ubiquitous computing systems
ThumbSpace: generalized one-handed input for touchscreen-based mobile devices

INTERACT'07 Proceedings of the 11th IFIP TC 13 international conference on Human-computer interaction
Gesture interaction in cooperation scenarios

CRIWG'09 Proceedings of the 15th international conference on Groupware: design, implementation, and use
Pressure-based menu selection for mobile devices

Proceedings of the 12th international conference on Human computer interaction with mobile devices and services
Sensing foot gestures from the pocket

UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Leveraging proprioception to make mobile phones more accessible to users with visual impairments

Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility
Multiple spatial sounds in hierarchical menu navigation for visually impaired computer users

International Journal of Human-Computer Studies
Touching the void: gestures for auditory interfaces

Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction
Spindex (Speech Index) Improves Auditory Menu Acceptance and Navigation Performance

ACM Transactions on Accessible Computing (TACCESS)
Air pointing: Design and evaluation of spatial target acquisition with and without visual feedback

International Journal of Human-Computer Studies
Theories, methods and current research on emotions in library and information science, information retrieval and human-computer interaction

Information Processing and Management: an International Journal
Gaze-directed hands-free interface for mobile interaction

HCII'11 Proceedings of the 14th international conference on Human-computer interaction: interaction techniques and environments - Volume Part II
Body, movement, gesture & tactility in interaction with mobile devices

Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services
Using the user's point of view for interaction on mobile devices

23rd French Speaking Conference on Human-Computer Interaction
Tap control for headphones without sensors

Proceedings of the 24th annual ACM symposium on User interface software and technology
6th senses for everyone!: the value of multimodal feedback in handheld navigation aids

ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Extending mobile user ambient awareness for nomadic text entry

Proceedings of the 23rd Australian Computer-Human Interaction Conference
Multimodal human computer interaction: a survey

ICCV'05 Proceedings of the 2005 international conference on Computer Vision in Human-Computer Interaction
Effects of auditory feedback for augmenting the act of writing

AH '12 Proceedings of the 3rd Augmented Human International Conference
Simulator sickness in mobile spatial sound spaces

CMMR/ICAD'09 Proceedings of the 6th international conference on Auditory Display
Aural browsing on-the-go: listening-based back navigation in large web architectures

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Exploring user motivations for eyes-free interaction on mobile devices

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Multimodal interactions for expressive interfaces

EISE'09 Proceedings of the First international conference on Expressive Interactions for Sustainability and Empowerment
“Spindex” (Speech Index) Enhances Menus on Touch Screen Devices with Tapping, Wheeling, and Flicking

ACM Transactions on Computer-Human Interaction (TOCHI)
Headphone taps: a simple technique to add input function to regular headphones

MobileHCI '12 Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services companion
Eyes-free interaction with free-hand gestures and auditory menus

International Journal of Human-Computer Studies
Understanding palm-based imaginary interfaces: the role of visual and tactile cues when browsing

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Conductive rubber electrodes for earphone-based eye gesture input interface

Proceedings of the 2013 International Symposium on Wearable Computers

Quantified Score

Hi-index	0.01

Visualization

Abstract

Mobile and wearable computers present input/output prob-lems due to limited screen space and interaction techniques. When mobile, users typically focus their visual attention on navigating their environment - making visually demanding interface designs hard to operate. This paper presents two multimodal interaction techniques designed to overcome these problems and allow truly mobile, 'eyes-free' device use. The first is a 3D audio radial pie menu that uses head gestures for selecting items. An evaluation of a range of different audio designs showed that egocentric sounds re-duced task completion time, perceived annoyance, and al-lowed users to walk closer to their preferred walking speed. The second is a sonically enhanced 2D gesture recognition system for use on a belt-mounted PDA. An evaluation of the system with and without audio feedback showed users' ges-tures were more accurate when dynamically guided by au-dio-feedback. These novel interaction techniques demon-strate effective alternatives to visual-centric interface de-signs on mobile devices.