Using dual cascading learning frameworks for image indexing

• Authors: Joo-Hwee Lim;Jesse S. Jin
• Venue: VIP '05 Proceedings of the Pan-Sydney area workshop on Visual information processing
• Year: 2004

To bridge the semantic gap in content-based image retrieval, detecting meaningful visual entities (e.g. faces, sky, foliage, buildings etc) in image content and classifying images into semantic categories based on trained pattern classifiers have become active research trends. In this paper, we present dual cascading learning frameworks that extract and combine intra-image and inter-class semantics for image indexing and retrieval.In the supervised learning version, support vector detectors are trained on semantic support regions without image segmentation. The reconciled and aggregated detection-based indexes then serve as input for support vector learning of image classifiers to generate class-relative image indexes. During retrieval, similarities based on both indexes are combined to rank images.In the unsupervised learning approach, image classifiers are first trained on local image blocks from a small number of labeled images. Then local semantic patterns are discovered from clustering the image blocks with high classification output. Training samples are induced from cluster memberships for support vector learning to form local semantic pattern detectors. During retrieval, similarities based on local class pattern indexes and discovered pattern indexes are combined to rank images.Query-by-example experiments on 2400 unconstrained consumer photos with 16 semantic queries show that the combined matching aproaches are better than matching with single indexes. Both the supervised semantics design and the semantics discovery approaches also outperformed the linear fusion of color and texture features significantly in average precisions by 55% and 37% respectively.