Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary
ECCV '02 Proceedings of the 7th European Conference on Computer Vision-Part IV
Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
The Journal of Machine Learning Research
Multiple Bernoulli relevance models for image and video annotation
CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
| Hi-index | 12.05 |
Video annotation has become an important topic to support multimedia information retrieval. Video content analysis using low-level features cannot reduce the gap between low-level features and high level semantic concept. In this study, we propose an approach which combines visual features extracted from visual track of video and keywords extracted from speech transcripts of audio track. We construct a predictive model using hierarchical Dirichlet process mixture model. In the hierarchical model, one more layer is added to exploit sharing visual feature distributions among frames and use the shared information to enhance model learning. At top level the visual features in the groups are shared appropriately by imposing a prior correlation. At the bottom level each visual feature and associated annotation are modeled with mixture distributions. The leaned predictive model allows us to compute a conditional likelihood over words which are used to predict the most likely annotation words for the testing sample. The model achieves high accuracy in video annotation than the model without using hierarchy.