VoiceLabel: using speech to label mobile sensor data

Authors:
Susumu Harada;Jonathan Lester;Kayur Patel;T. Scott Saponas;James Fogarty;James A. Landay;Jacob O. Wobbrock
Affiliations:
University of Washington, Seattle, WA, USA;University of Washington, Seattle, WA, USA;University of Washingotn, Seattle, WA, USA;University of Washington, Seattle, WA, USA;University of Washington, Seattle, WA, USA;University of Washington, Seattle, WA, USA;University of Washington, Seattle, WA, USA
Venue:
ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
Year:
2008

Citing 10
Cited 2

System architecture directions for networked sensors

ASPLOS IX Proceedings of the ninth international conference on Architectural support for programming languages and operating systems
Speaker-Dependent Speech Recognition Based on Phone-Like Units Models - Application to Voice Dialing

ICASSP '97 Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97)-Volume 2 - Volume 2
CueTIP: a mixed-initiative interface for correcting handwriting errors

UIST '06 Proceedings of the 19th annual ACM symposium on User interface software and technology
The vocal joystick: a voice-based human-computer interface for individuals with motor impairments

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
Smart headphones: enhancing auditory awareness through robust speech detection and source localization

ICASSP '01 Proceedings of the Acoustics, Speech, and Signal Processing, 2001. on IEEE International Conference - Volume 05
Activity sensing in the wild: a field trial of ubifit garden

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A hybrid discriminative/generative approach for modeling human activities

IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
A long-term evaluation of sensing modalities for activity recognition

UbiComp '07 Proceedings of the 9th international conference on Ubiquitous computing
A practical approach to recognizing physical activities

PERVASIVE'06 Proceedings of the 4th international conference on Pervasive Computing
Practical metropolitan-scale positioning for GSM phones

UbiComp'06 Proceedings of the 8th international conference on Ubiquitous Computing

A multimodal labeling interface for wearable computing

Proceedings of the 15th international conference on Intelligent user interfaces
Verbal control of mathematical tools for simulation and virtual environments

Proceedings of the 2010 Summer Computer Simulation Conference

Quantified Score

Hi-index	0.00

Visualization

Abstract

Many mobile machine learning applications require collecting and labeling data, and a traditional GUI on a mobile device may not be an appropriate or viable method for this task. This paper presents an alternative approach to mobile labeling of sensor data called VoiceLabel. VoiceLabel consists of two components: (1) a speech-based data collection tool for mobile devices, and (2) a desktop tool for offline segmentation of recorded data and recognition of spoken labels. The desktop tool automatically analyzes the audio stream to find and recognize spoken labels, and then presents a multimodal interface for reviewing and correcting data labels using a combination of the audio stream, the system's analysis of that audio, and the corresponding mobile sensor data. A study with ten participants showed that VoiceLabel is a viable method for labeling mobile sensor data. VoiceLabel also illustrates several key features that inform the design of other data labeling tools.