The vocal joystick: a voice-based human-computer interface for individuals with motor impairments

Authors:
Jeff A. Bilmes;Xiao Li;Jonathan Malkin;Kelley Kilanski;Richard Wright;Katrin Kirchhoff;Amarnag Subramanya;Susumu Harada;James A. Landay;Patricia Dowden;Howard Chizeck
Affiliations:
University of Washington, Seattle, WA;University of Washington, Seattle, WA;University of Washington, Seattle, WA;University of Washington, Seattle, WA;University of Washington, Seattle, WA;University of Washington, Seattle, WA;University of Washington, Seattle, WA;University of Washington, Seattle, WA;University of Washington, Seattle, WA;University of Washington, Seattle, WA;University of Washington, Seattle, WA
Venue:
HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
Year:
2005

Citing 5
Cited 22

Dasher—a data entry interface using continuous gestures and language models

UIST '00 Proceedings of the 13th annual ACM symposium on User interface software and technology
Voice as sound: using non-verbal voice input for interactive control

Proceedings of the 14th annual ACM symposium on User interface software and technology
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
Spoken Language Processing: A Guide to Theory, Algorithm, and System Development

Spoken Language Processing: A Guide to Theory, Algorithm, and System Development
Interaction techniques using prosodic features of speech and audio localization

Proceedings of the 10th international conference on Intelligent user interfaces

The vocal joystick:: evaluation of voice-based cursor control techniques

Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility
Non-speech input and speech recognition for real-time control of computer games

Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility
Voicedraw: a hands-free voice-driven drawing application for people with motor impairments

Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility
Demo of VJ-Voicebot: control of robotic arm with the Vocal Joystick

Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility
Humming control interface for hand-held devices

Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility
VoicePen: augmenting pen input with simultaneous non-linguisitic vocalization

Proceedings of the 9th international conference on Multimodal interfaces
Keysurf: a character controlled browser for people with physical disabilities

Proceedings of the 17th international conference on World Wide Web
VoiceLabel: using speech to label mobile sensor data

ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
The VoiceBot: a voice controlled robot arm

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Longitudinal study of people learning to use continuous voice-based cursor control

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Testing inertial sensor performance as hands-free human-computer interface

WSEAS Transactions on Computers
Pitch in non-verbal vocal input

ACM SIGACCESS Accessibility and Computing
User experience of speech controlled media center for physically disabled users

Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era
Measuring throughput of the headjoystick human-computer interface

SMO'09 Proceedings of the 9th WSEAS international conference on Simulation, modelling and optimization
Experimental measurement of information transfer rate of the inertial sensor based human-computer interface for the disabled

WSEAS TRANSACTIONS on SYSTEMS
A comparative longitudinal study of non-verbal mouse pointer

INTERACT'07 Proceedings of the 11th IFIP TC 13 international conference on Human-computer interaction - Volume Part II
Investigating Grid-Based Navigation: The Impact of Physical Disability

ACM Transactions on Accessible Computing (TACCESS)
Design and evaluation of a non-verbal voice-controlled cursor for point-and-click tasks

Proceedings of the 4th International Convention on Rehabilitation Engineering & Assistive Technology
The Vocal Joystick Engine v1.0

Computer Speech and Language
An interactive Whistle-to-Music composing system based on transcription, variation and chords generation

Multimedia Tools and Applications
A tongue input device for creating conversations

Proceedings of the 24th annual ACM symposium on User interface software and technology
Desktop access with non-verbal sound input

i-CREATe '11 Proceedings of the 5th International Conference on Rehabilitation Engineering & Assistive Technology

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a novel voice-based human-computer interface designed to enable individuals with motor impairments to use vocal parameters for continuous control tasks. Since discrete spoken commands are ill-suited to such tasks, our interface exploits a large set of continuous acoustic-phonetic parameters like pitch, loudness, vowel quality, etc. Their selection is optimized with respect to automatic recognizability, communication bandwidth, learnability, suitability, and ease of use. Parameters are extracted in real time, transformed via adaptation and acceleration, and converted into continuous control signals. This paper describes the basic engine, prototype applications (in particular, voice-based web browsing and a controlled trajectory-following task), and initial user studies confirming the feasibility of this technology.