Detecting sweethearting in retail surveillance videos

  • Authors:
  • Quanfu Fan;Akira Yanagawa;Russell Bobbitt; Yun Zhai;Rick Kjeldsen;Sharath Pankanti;Arun Hampapur

  • Affiliations:
  • IBM T. J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY 10532, USA;IBM T. J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY 10532, USA;IBM T. J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY 10532, USA;IBM T. J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY 10532, USA;IBM T. J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY 10532, USA;IBM T. J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY 10532, USA;IBM T. J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY 10532, USA

  • Venue:
  • ICASSP '09 Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

A significant portion of retail shrink is attributed to employees and occurs around the point of sale (POS). In this paper, we target a major type of retail fraud in surveillance videos, known as sweethearting (or fake scan), where a cashier intentionally fails to enter one or more items into the transaction in an attempt to get free merchandise for the customer. We first develop a motion-based algorithm to identify video segments as candidates for primitive events at the POS. We then apply spatio-temporal features to recognize true primitive events from the candidates and prune those falsely alarmed. In particular, we learn location-aware event models by Multiple-Instance Learning to address the location-sensitive issues that appear in our problem. Finally, we validate the entire transaction by combining primitive events according to temporal ordering constraints. We demonstrate the effectiveness of our approach on data captured from a real grocery store.