Parallel Lasso for Large-Scale Video Concept Detection

• Authors: Bo Geng;Yangxi Li;Dacheng Tao;Meng Wang;Zheng-Jun Zha;Chao Xu
• Venue: IEEE Transactions on Multimedia
• Year: 2012

Existing video concept detectors are generally built upon the kernel based machine learning techniques, e.g., support vector machines, regularized least squares, and logistic regression, just to name a few. However, in order to build robust detectors, the learning process suffers from the scalability issues including the high-dimensional multi-modality visual features and the large-scale keyframe examples. In this paper, we propose parallel lasso (Plasso) by introducing the parallel distributed computation to significantly improve the scalability of lasso (the $l_1$ regularized least squares). We apply the parallel incomplete Cholesky factorization to approximate the covariance statistics in the preprocess step, and the parallel primal-dual interior-point method with the Sherman-Morrison-Woodbury formula to optimize the model parameters. For a dataset with $n$ samples in a $d$-dimensional space, compared with lasso, Plasso significantly reduces complexities from the original $O(d^3)$ for computational time and $O(d^2)$ for storage space to $O(h^2d/m)$ and $O(hd/m)$ , respectively, if the system has $m$ processors and the reduced dimension $h$ is much smaller than the original dimension $d$ . Furthermore, we develop the kernel extension of the proposed linear algorithm with the sample reweighting schema, and we can achieve similar time and space complexity improvements [time complexity from $O(n^3)$ to $O(h^2n/m)$ and the space complexity from $O(n^2)$ to $O(hn/m)$, for a dataset with $n$ training examples]. Experimental results on TRECVID video concept detection challenges suggest that the proposed method can obtain significant time and space savings for training effective detectors with limited communication overhead.