Physical activity recognition based on motion in images acquired by a wearable camera

  • Authors:

  • Hong Zhang;Lu Li;Wenyan Jia;John D. Fernstrom;Robert J. Sclabassi;Zhi-Hong Mao;Mingui Sun

  • Affiliations:

  • Image Processing Center, Beihang University, Beijing 100191, China;Image Processing Center, Beihang University, Beijing 100191, China and Department of Neurosurgery, University of Pittsburgh, PA 15213, USA;Department of Neurosurgery, University of Pittsburgh, PA 15213, USA;Department of Psychiatry, University of Pittsburgh, PA 15261, USA;Department of Neurosurgery, University of Pittsburgh, PA 15213, USA;Department of Electrical and Computer Engineering, University of Pittsburgh, PA 15261, USA;Department of Neurosurgery, University of Pittsburgh, PA 15213, USA and Department of Electrical and Computer Engineering, University of Pittsburgh, PA 15261, USA

  • Venue:
  • Neurocomputing
  • Year:
  • 2011

Quantified Score

Hi-index 0.01

Visualization

Abstract

A new technique to extract and evaluate physical activity patterns from image sequences captured by a wearable camera is presented in this paper. Unlike standard activity recognition schemes, the video data captured by our device do not include the wearer him/herself. The physical activity of the wearer, such as walking or exercising, is analyzed indirectly through the camera motion extracted from the acquired video frames. Two key tasks, pixel correspondence identification and motion feature extraction, are studied to recognize activity patterns. We utilize a multiscale approach to identify pixel correspondences. When compared with the existing methods such as the good features detector and the speed-up robust feature (SURF) detector, our technique is more accurate and computationally efficient. Once the pixel correspondences are determined which define representative motion vectors, we build a set of activity pattern features based on motion statistics in each frame. Finally, the physical activity of the person wearing a camera is determined according to the global motion distribution in the video. Our algorithms are tested using different machine learning techniques such as the K-nearest neighbor (KNN), naive Bayesian and support vector machine (SVM). The results show that many types of physical activities can be recognized from field acquired real-world video. Our results also indicate that, with a design of specific motion features in the input vectors, different classifiers can be used successfully with similar performances.