Enhanced representation and multi-task learning for image annotation

  • Authors:

  • Alexander Binder;Wojciech Samek;Klaus-Robert MüLler;Motoaki Kawanabe

  • Affiliations:

  • Machine Learning Group, Berlin Institute of Technology (TU Berlin), Marchstrasse 23, 10587 Berlin, Germany and Fraunhofer Institute FIRST, Kekuléstr. 7, 12489 Berlin, Germany;Machine Learning Group, Berlin Institute of Technology (TU Berlin), Marchstrasse 23, 10587 Berlin, Germany and Fraunhofer Institute FIRST, Kekuléstr. 7, 12489 Berlin, Germany;Machine Learning Group, Berlin Institute of Technology (TU Berlin), Marchstrasse 23, 10587 Berlin, Germany and Bernstein Focus: Neurotechnology Berlin, 10587 Berlin, Germany and Department of Brai ...;ATR Brain Information Communication Research Laboratory Group, 2-2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, Japan and Fraunhofer Institute FIRST, Kekuléstr. 7, 12489 Berlin, German ...

  • Venue:
  • Computer Vision and Image Understanding
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper we propose a novel biased random sampling strategy for image representation in Bag-of-Words models. We evaluate its impact on the feature properties and the ranking quality for a set of semantic concepts and show that it improves performance of classifiers in image annotation tasks and increases the correlation between kernels and labels. As second contribution we propose a method called Output Kernel Multi-Task Learning (MTL) to improve ranking performance by transfer information between classes. The main advantages of output kernel MTL are that it permits asymmetric information transfer between tasks and scales to training sets of several thousand images. We give a theoretical interpretation of the method and show that the learned contributions of source tasks to target tasks are semantically consistent. Both strategies are evaluated on the ImageCLEF PhotoAnnotation dataset. Our best visual result which used the MTL method was ranked first according to mean Average Precision (mAP) within the purely visual submissions in the ImageCLEF 2011 PhotoAnnotation Challenge. Our multi-modal submission achieved the first rank by mAP among all submissions in the same competition.