Augmenting mobile city-view image retrieval with context-rich user-contributed photos

Authors:
Yin-Hsi Kuo;Wen-Yu Lee;Winston H. Hsu;Wen-Huang Cheng
Affiliations:
National Taiwan University and Academia Sinica, Taipei, Taiwan Roc;National Taiwan University, Taipei, Taiwan Roc;National Taiwan University, Taipei, Taiwan Roc;Academia Sinica, Taipei, Taiwan Roc
Venue:
MM '11 Proceedings of the 19th ACM international conference on Multimedia
Year:
2011

Citing 8
Cited 1

Video Google: A Text Retrieval Approach to Object Matching in Videos

ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
Inverted files for text search engines

ACM Computing Surveys (CSUR)
Scalable Recognition with a Vocabulary Tree

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
Beyond GPS: determining the camera viewing direction of a geotagged image

Proceedings of the international conference on Multimedia
GPS, compass, or camera?: investigating effective mobile sensors for automatic search-based image annotation

Proceedings of the international conference on Multimedia
Avoiding confusing features in place recognition

ECCV'10 Proceedings of the 11th European conference on Computer vision: Part I
Geotagging in multimedia and computer vision--a survey

Multimedia Tools and Applications
City-scale landmark identification on mobile devices

CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition

City-view image retrieval leveraging check-in data

Proceedings of the 2nd ACM international workshop on Geotagging and its applications in multimedia

Quantified Score

Hi-index	0.00

Visualization

Abstract

With the growth of mobile devices, the needs for location-based services are emerging. Taking the advantage of the GPS information, we can roughly estimate a user's location. However, it is necessary to leverage extra information (e.g., photos) to precisely locate the object of interest through mobile devices for further applications such as mobile search. Users can simply take a picture (with GPS enabled) of an interesting target to retrieve the building information. Therefore, the raise of real-time building recognition or retrieval system becomes a challenging problem. The most recent approaches are to recognize buildings by the street-view images; however, the query photos from mobile devices usually contain different lighting conditions. In order to provide a more robust city-view image retrieval system, we propose to augment the visual diversity of database images by integrating the context-rich user-contributed photos from social media. Preliminary experimental results show that the street-view images can provide different angles of the target whereas the user-contributed photos can enhance the diversity of the target. Besides, for the real-time retrieval system, we also combine both visual and GPS constraints in the retrieval process on inverted indexing so that we can achieve a real-time retrieval system.