Interactive activity recognition and prompting to assist people with cognitive disabilities

  • Authors:
  • Yi Chu;Young Chol Song;Richard Levinson;Henry Kautz

  • Affiliations:
  • Department of Computer Science, University of Rochester, Rochester, NY 14627, USA;Department of Computer Science, University of Rochester, Rochester, NY 14627, USA;Attention Control Systems, 650 Castro Street, PMB 120-197, Mountain View, CA 94041, USA. URL: www.brainaid.com;Department of Computer Science, University of Rochester, Rochester, NY 14627, USA

  • Venue:
  • Journal of Ambient Intelligence and Smart Environments - Home-based Health and Wellness Measurement and Monitoring
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper presents a model of interactive activity recognition and prompting for use in an assistive system for persons with cognitive disabilities. The system can determine the user's state by interpreting sensor data and/or by explicitly querying the user, and can prompt the user to begin, resume, or end tasks. The objective of the system is to help the user maintain a daily schedule of activities while minimizing interruptions from questions or prompts. The model is built upon an option-based hierarchical POMDP. Options can be programmed and customized to specify complex routines for prompting or questioning.The paper proposes a heuristic approach to solving the POMDP based on a dual control algorithm using selective-inquiry that can appeal for help from the user explicitly when the sensor data is ambiguous. The dual control algorithm is working effectively in the unified control model which features the adaptive option and robust state estimation. Simulation results show that the unified dual control model achieves the best performance and efficiency comparing with various alternatives. To further demonstrate the system's performance, lab experiments have been carried out with volunteer actors performing a series of carefully designed scenarios with different kinds of interruption cases. The results show that the system is able to successfully guide the agent through the sample schedule by delivering correct prompts while efficiently dealing with ambiguous situations.