Generating natural-language narratives from activity recognition with spurious classification pruning

  • Authors:
  • Thomas Phan

  • Affiliations:
  • Samsung R&D Center, San Jose, CA

  • Venue:
  • Proceedings of the Third International Workshop on Sensing Applications on Mobile Phones
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Continuous smartphone sensing offers the opportunity to perform recognition and logging of the user's activities, but a key challenge remains in conveying these activities in a manner that is both understandable by humans and useful for downstream consuming applications. We explore the viability of representing activity recognition through automatically-generated natural language English text, allowing the user to understand the activity recognition output and other software to operate on it, such as with text-to-speech software or an information retrieval search engine. To create narratives with smoother transitions and to help intelligently trigger the GPS in a power-conserving manner, we implemented a scheme called Spurious Sequential Classification Pruning that we show reduces real-time misclassifications by 76% and GPS requests by 38%. We describe a complete system for activity recognition and narrative generation and discuss its end-to-end operation.