Semantic assessment of shopping behavior using trajectories, shopping related actions, and context information

  • Authors:

  • M. C. Popa;L. J. M. Rothkrantz;C. Shan;T. Gritti;P. Wiggers

  • Affiliations:

  • Section of Interactive Intelligence, Department of Intelligence Systems, Delft University of Technology, Mekelweg 4, 2628 CD, Delft, The Netherlands and Video and Image Processing Department, Phil ...;Section of Interactive Intelligence, Department of Intelligence Systems, Delft University of Technology, Mekelweg 4, 2628 CD, Delft, The Netherlands and Sensor Technology, SEWACO Department, Nethe ...;Video and Image Processing Department, Philips Research, HTC 36, 5656 AE, Eindhoven, The Netherlands;Video and Image Processing Department, Philips Research, HTC 36, 5656 AE, Eindhoven, The Netherlands;Section of Interactive Intelligence, Department of Intelligence Systems, Delft University of Technology, Mekelweg 4, 2628 CD, Delft, The Netherlands

  • Venue:
  • Pattern Recognition Letters
  • Year:
  • 2013

Quantified Score

Hi-index 0.10

Visualization

Abstract

Automatic understanding of customers' shopping behavior and acting according to their needs is relevant in the marketing domain and is attracting a lot of attention lately. In this work, we propose a multi-level framework for the automatic assessment of customers' shopping behavior. The low level input to the framework is obtained from different types of cameras, which are synchronized, facilitating efficient processing of information. A fish-eye camera is used for tracking people, while a high-definition one serves for the action recognition task. The experiments are performed on both laboratory and real-life recordings in a supermarket. From the video recordings, we extract features related to the spatio-temporal behavior of trajectories, the dynamics and the time spent in each region of interest (ROI) in the shop and regarding the customer-products interaction patterns. Next we analyze the shopping sequences using a Hidden Markov Model (HMM). We conclude that it is possible to accurately classify trajectories (93%), discriminate between different shopping related actions (91.6%), and recognize shopping behavioral types by means of our proposed reasoning model in 95% of the cases.