Learning Image-Text Associations

Authors:
Tao Jiang;Ah-Hwee Tan
Affiliations:
Nanyang Technological University, Singapore;Nanyang Technological University, Singapore
Venue:
IEEE Transactions on Knowledge and Data Engineering
Year:
2009

Citing 0
Cited 4

Multimedia data mining: state of the art and challenges

Multimedia Tools and Applications
An information fusion approach to integrate image annotation and text mining methods for geographic knowledge discovery

Expert Systems with Applications: An International Journal
Web page and image semi-supervised classification with heterogeneous information fusion

Journal of Information Science
Cross-language patent matching via an international patent classification-based concept bridge

Journal of Information Science

Quantified Score

Hi-index	0.00

Visualization

Abstract

Web information fusion can be defined as the problem of collating and tracking information related to specific topics on the World Wide Web. Whereas most existing work on web information fusion has focused on text-based multidocument summarization, this paper concerns the topic of image and text association, a cornerstone of cross-media web information fusion. Specifically, we present two learning methods for discovering the underlying associations between images and texts based on small training data sets. The first method based on vague transformation measures the information similarity between the visual features and the textual features through a set of predefined domain-specific information categories. Another method uses a neural network to learn direct mapping between the visual and textual features by automatically and incrementally summarizing the associated features into a set of information templates. Despite their distinct approaches, our experimental results on a terrorist domain document set show that both methods are capable of learning associations between images and texts from a small training data set.