WordNet: a lexical database for English
Communications of the ACM
Contrast-based image attention analysis by using fuzzy growing
MULTIMEDIA '03 Proceedings of the eleventh ACM international conference on Multimedia
Distinctive Image Features from Scale-Invariant Keypoints
International Journal of Computer Vision
Photo-to-Search: Using Camera Phones to Inquire of the Surrounding World
MDM '06 Proceedings of the 7th International Conference on Mobile Data Management
Scalable Recognition with a Vocabulary Tree
CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
Mobile information access: A study of emerging search behavior on the mobile Internet
ACM Transactions on the Web (TWEB)
Computer Vision and Image Understanding
Outdoors augmented reality on mobile phone using loxel-based visual feature organization
MIR '08 Proceedings of the 1st ACM international conference on Multimedia information retrieval
CrowdReranking: exploring multiple search engines for visual search reranking
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Image Description and Matching Scheme for Identical Image Searching
COMPUTATIONWORLD '09 Proceedings of the 2009 Computation World: Future Computing, Service Computation, Cognitive, Adaptive, Content, Patterns
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Understanding the semantic structure of noun phrase queries
ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
MindFinder: interactive sketch-based image search on millions of images
Proceedings of the international conference on Multimedia
Low latency image retrieval with progressive transmission of CHoG descriptors
Proceedings of the 2010 ACM multimedia workshop on Mobile cloud media computing
Interactive Image Search by Color Map
ACM Transactions on Intelligent Systems and Technology (TIST)
Discovering areas of interest with geo-tagged images and check-ins
Proceedings of the 20th ACM international conference on Multimedia
Mobile-based advertisement information retrieval from images and websites
Proceedings of the 20th ACM international conference on Multimedia
Local visual words coding for low bit rate mobile visual search
Proceedings of the 20th ACM international conference on Multimedia
Proceedings of the 20th ACM international conference on Multimedia
Multimedia search reranking: A literature survey
ACM Computing Surveys (CSUR)
Hi-index | 0.00 |
The traditional text-based visual search has not been sufficiently improved over the years to accommodate the new emerging demand of mobile users. While on the go, searching on one's phone is becoming pervasive. This paper presents an innovative application for mobile phone users to facilitate their visual search experience. By taking advantage of smart phone functionalities such as multi-modal and multi-touch interactions, users can more conveniently formulate their search intent, and thus search performance can be significantly improved. The system, called JIGSAW (Joint search with ImaGe, Speech, And Words), represents one of the first attempts to create an interactive and multi-modal mobile visual search application. The key of JIGSAW is the composition of an exemplary image query generated from the raw speech via multi-touch user interaction, as well as the visual search based on the exemplary image. Through JIGSAW, users can formulate their search intent in a natural way like playing a jigsaw puzzle on the phone screen: 1) a user speaks a natural sentence as the query, 2) the speech is recognized and transferred to text which is further decomposed to keywords through entity extraction, 3) the user selects preferred exemplary images that can visually represent his/her intent and composes a query image via multi-touch, and 4) the composite image is then used as a visual query to search similar images. We have deployed JIGSAW on a real-world phone system, evaluated the performance on one million images, and demonstrated that it is an effective complement to existing mobile visual search applications.