Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition
NLDB '02 Proceedings of the 6th International Conference on Applications of Natural Language to Information Systems-Revised Papers
Omnibase: Uniform Access to Heterogeneous Data for Question Answering
NLDB '02 Proceedings of the 6th International Conference on Applications of Natural Language to Information Systems-Revised Papers
Video Google: A Text Retrieval Approach to Object Matching in Videos
ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
VideoQA: question answering on news video
MULTIMEDIA '03 Proceedings of the eleventh ACM international conference on Multimedia
Labeling images with a computer game
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
CVPRW '04 Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 12 - Volume 12
A picture is worth a thousand keywords: image-based object search on a mobile platform
CHI '05 Extended Abstracts on Human Factors in Computing Systems
Automated Question Answering: Review of the Main Approaches
ICITA '05 Proceedings of the Third International Conference on Information Technology and Applications (ICITA'05) Volume 2 - Volume 02
Photo-to-search: using multimodal queries to search the web from mobile devices
Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval
Scalable Recognition with a Vocabulary Tree
CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
Question Similarity Calculation for FAQ Answering
SKG '07 Proceedings of the Third International Conference on Semantics, Knowledge and Grid
Multimodal question answering for mobile devices
Proceedings of the 13th international conference on Intelligent user interfaces
ACCV'07 Proceedings of the 8th Asian conference on Computer vision - Volume Part I
EMMCVPR'07 Proceedings of the 6th international conference on Energy minimization methods in computer vision and pattern recognition
Searching the web with mobile images for location recognition
CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
PCA-SIFT: a more distinctive representation for local image descriptors
CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
Top-points as interest points for image matching
ECCV'06 Proceedings of the 9th European conference on Computer Vision - Volume Part I
From text question-answering to multimedia QA on web-scale media resources
LS-MMRM '09 Proceedings of the First ACM workshop on Large-scale multimedia retrieval and mining
Query expansion for hash-based image object retrieval
MM '09 Proceedings of the 17th ACM international conference on Multimedia
Video reference: question answering on YouTube
MM '09 Proceedings of the 17th ACM international conference on Multimedia
Web image interpretation: semi-supervised mining annotated words
ICME'09 Proceedings of the 2009 IEEE international conference on Multimedia and Expo
Exploring large scale data for multimedia QA: an initial study
Proceedings of the ACM International Conference on Image and Video Retrieval
VizWiz: nearly real-time answers to visual questions
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Modeling betweenness for question answering
ESAIR '10 Proceedings of the third workshop on Exploiting semantic annotations in information retrieval
Boosting image object retrieval and indexing by automatically discovered pseudo-objects
Journal of Visual Communication and Image Representation
Interactive inquiry for object of interest in video playback by motion-augmented graph cut
Proceedings of the international conference on Multimedia
The InfoAlbum image centric information collection
Proceedings of the International Conference on Web Intelligence, Mining and Semantics
Multimedia answering: enriching text QA with media information
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Flower information retrieval using color feature and location-based system
AICT'11 Proceedings of the 2nd international conference on Applied informatics and computing theory
RFID-based interactive multimedia system for the children
Multimedia Tools and Applications
Snap-and-ask: answering multimodal question by naming visual instance
Proceedings of the 20th ACM international conference on Multimedia
Building Multi-Modal Relational Graphs for Multimedia Retrieval
International Journal of Multimedia Data Engineering & Management
Hi-index | 0.00 |
Photo-based question answering is a useful way of finding information about physical objects. Current question answering (QA) systems are text-based and can be difficult to use when a question involves an object with distinct visual features. A photo-based QA system allows direct use of a photo to refer to the object. We develop a three-layer system architecture for photo-based QA that brings together recent technical achievements in question answering and image matching. The first, template-based QA layer matches a query photo to online images and extracts structured data from multimedia databases to answer questions about the photo. To simplify image matching, it exploits the question text to filter images based on categories and keywords. The second, information retrieval QA layer searches an internal repository of resolved photo-based questions to retrieve relevant answers. The third, human-computation QA layer leverages community experts to handle the most difficult cases. A series of experiments performed on a pilot dataset of 30,000 images of books, movie DVD covers, grocery items, and landmarks demonstrate the technical feasibility of this architecture. We present three prototypes to show how photo-based QA can be built into an online album, a text-based QA, and a mobile application.