SoundWave: using the doppler effect to sense gestures

Authors:
Sidhant Gupta;Daniel Morris;Shwetak Patel;Desney Tan
Affiliations:
Microsoft Research, Redmond & University of Washington, Seattle, Washington, United States;Microsoft Research, Redmond, Washington, United States;Microsoft Research, Redmond & University of Washington, Seattle, Washington, United States;Microsoft Research, Redmond, Washington, United States
Venue:
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Year:
2012

Citing 3
Cited 6

The magic carpet: physical sensing for immersive environments

CHI EA '97 CHI '97 Extended Abstracts on Human Factors in Computing Systems
One-handed gesture recognition using ultrasonic Doppler sonar

ICASSP '09 Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing
Sonar-based measurement of user presence and attention

Proceedings of the 11th international conference on Ubiquitous computing

Spartacus: spatially-aware interaction for mobile devices through energy-efficient audio sensing

Proceeding of the 11th annual international conference on Mobile systems, applications, and services
DopLink: using the doppler effect for multi-device interaction

Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing
Ultrasound-based movement sensing, gesture-, and context-recognition

Proceedings of the 2013 International Symposium on Wearable Computers
1st workshop on human factors and activity recognition in healthcare, wellness and assisted living: recognise2interact

Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication
Whole-home gesture recognition using wireless signals

Proceedings of the 19th annual international conference on Mobile computing & networking
Touch & activate: adding interactivity to existing objects using active acoustic sensing

Proceedings of the 26th annual ACM symposium on User interface software and technology

Quantified Score

Hi-index	0.01

Visualization

Abstract

Gesture is becoming an increasingly popular means of interacting with computers. However, it is still relatively costly to deploy robust gesture recognition sensors in existing mobile platforms. We present SoundWave, a technique that leverages the speaker and microphone already embedded in most commodity devices to sense in-air gestures around the device. To do this, we generate an inaudible tone, which gets frequency-shifted when it reflects off moving objects like the hand. We measure this shift with the microphone to infer various gestures. In this note, we describe the phenomena and detection algorithm, demonstrate a variety of gestures, and present an informal evaluation on the robustness of this approach across different devices and people.