Voice augmented manipulation: using paralinguistic information to manipulate mobile devices

Authors:
Daisuke Sakamoto;Takanori Komatsu;Takeo Igarashi
Affiliations:
The University of Tokyo, Bunkyo-ku, Tokyo, Japan;Meiji University, Nakano-ku, Tokyo, Japan;The University of Tokyo, Bunkyo-ku, Tokyo, Japan
Venue:
Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services
Year:
2013

Citing 29
Cited 0

A comparison of voice controlled and mouse controlled web browsing

Assets '00 Proceedings of the fourth international ACM conference on Assistive technologies
Voice as sound: using non-verbal voice input for interactive control

Proceedings of the 14th annual ACM symposium on User interface software and technology
“Put-that-there”: Voice and gesture at the graphics interface

SIGGRAPH '80 Proceedings of the 7th annual conference on Computer graphics and interactive techniques
The production and recognition of emotions in speech: features and algorithms

International Journal of Human-Computer Studies - Application of affective computing in human—Computer interaction
Speech-based cursor control: a study of grid-based solutions

Assets '04 Proceedings of the 6th international ACM SIGACCESS conference on Computers and accessibility
Collapse-to-zoom: viewing web pages on small screen devices by interactively removing irrelevant content

Proceedings of the 17th annual ACM symposium on User interface software and technology
Wired for Speech: How Voice Activates and Advances the Human-Computer Relationship

Wired for Speech: How Voice Activates and Advances the Human-Computer Relationship
Speech pen: predictive handwriting based on ambient multimodal recognition

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The vocal joystick:: evaluation of voice-based cursor control techniques

Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility
Blui: low-cost localized blowable user interfaces

Proceedings of the 20th annual ACM symposium on User interface software and technology
Voicedraw: a hands-free voice-driven drawing application for people with motor impairments

Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility
Map navigation with mobile devices: virtual versus physical movement with and without visual context

Proceedings of the 9th international conference on Multimodal interfaces
VoicePen: augmenting pen input with simultaneous non-linguisitic vocalization

Proceedings of the 9th international conference on Multimodal interfaces
Rubbing and tapping for precise and rapid selection on touch-screen displays

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
SideSight: multi-"touch" interaction around small devices

Proceedings of the 21st annual ACM symposium on User interface software and technology
Hands-free, speech-based navigation during dictation: difficulties, consequences, and solutions

Human-Computer Interaction
"Move the couch where?": developing an augmented reality multimodal interface

ISMAR '06 Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality
The VoiceBot: a voice controlled robot arm

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Longitudinal study of people learning to use continuous voice-based cursor control

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
An evaluation of one-handed techniques for multiple-target selection

CHI '09 Extended Abstracts on Human Factors in Computing Systems
Designing the User Interface: Strategies for Effective Human-Computer Interaction

Designing the User Interface: Strategies for Effective Human-Computer Interaction
GraspZoom: zooming and scrolling control model for single-handed mobile interaction

Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services
Interactions in the air: adding further depth to interactive tabletops

Proceedings of the 22nd annual ACM symposium on User interface software and technology
Clutch-free panning and integrated pan-zoom control on touch-sensitive surfaces: the cyclostar approach

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Semi-automatic zooming for mobile map navigation

Proceedings of the 12th international conference on Human computer interaction with mobile devices and services
Gesture search: a tool for fast mobile data access

UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Experimental analysis of touch-screen gesture designs in mobile environments

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
CHANTI: predictive text entry using non-verbal vocal input

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Speech augmented multitouch interaction patterns

Proceedings of the 16th European Conference on Pattern Languages of Programs

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose a technique called voice augmented manipulation (VAM) for augmenting user operations in a mobile environment. This technique augments user interactions on mobile devices, such as finger gestures and button pressing, with voice. For example, when a user makes a finger gesture on a mobile phone and voices a sound into it, the operation will continue until stops making the sound or makes another finger gesture. The VAM interface also provides a button-based interface, and the function connected to the button is augmented by voiced sounds. Two experiments verified the effectiveness of the VAM technique and showed that repeated finger gestures significantly decreased compared to current touch-input techniques, suggesting that VAM is useful in supporting user control in a mobile environment.