Combining image and text features: a hybrid approach to mobile book spine recognition

  • Authors:

  • Sam S. Tsai;David Chen;Huizhong Chen;Cheng-Hsin Hsu;Kyu-Han Kim;Jatinder P. Singh;Bernd Girod

  • Affiliations:

  • Stanford University, Stanford, CA, USA;Stanford University, Stanford, CA, USA;Stanford University, Stanford, CA, USA;National Tsing Hua University, Hsinchu, Taiwan Roc;HP Laboratories, Palo Alto, CA, USA;Deutsche Telekom R&D Laboratories USA, Los Altos, CA, USA;Stanford University, Stanford, CA, USA

  • Venue:
  • MM '11 Proceedings of the 19th ACM international conference on Multimedia
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Despite the successful use of local image features for large-scale object recognition, they are not effective in recognizing book spines on bookshelves. This is because some book spines contain only text components that do not yield distinguishing image features. To overcome this issue, we develop a new approach that combines a text-based spine recognition pipeline with an image feature-based spine recognition pipeline. The text within the book spine image is recognized and used as keywords to search a book spine text database. The image features of the book spine image are searched through a book spine image database. The search results of the two approaches are then carefully combined to form the final result. We implement the proposed hybrid book recognition pipeline used in a book inventory management system, and conduct extensive experiments to evaluate its performance. The experimental results show that while text-based or image feature-based systems only achieve a recall of 72%, the proposed hybrid system achieves a recall of ~91%.