Search strategies in multimodal image retrieval

  • Authors:
  • Stina Westman;Antti Lustila;Pirkko Oittinen

  • Affiliations:
  • Helsinki University of Technology, TKK Finland;Helsinki University of Technology, TKK Finland;Helsinki University of Technology, TKK Finland

  • Venue:
  • Proceedings of the second international symposium on Information interaction in context
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper reports on a study on search strategies in multimodal image retrieval. We analyzed the queries and search tactics employed by image journalism professionals and non-professionals in a user test. Transaction log data show that searchers are able to combine up to four query modes into a query. Most queries combined at least two of the modes (text, color, sketch, quality, and category). Task type was shown to affect the choice of which modes to employ. Known item and data search tasks led to queries combining text, color and category modes. Visually cued tasks resulted in searches combining several content-based and textual modes. Conceptual tasks led to a large number of queries by text and category only. User background also significantly affected the types of queries constructed. Professionals used the color mode more whereas non-professionals drew more sketches. Non-professionals were more likely to switch query modes whereas professionals edited the content of their queries. We described the search processes with Markov models and maximal repeating patterns. Common patterns and probable transitions dealt with querying, inspecting result images and saving them into the workspace or iterating queries of the same type. The results indicate a need to support multimodal image query formulation.