Semi-supervised sparse metric learning using alternating linearization optimization
Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining
Learning sparse Gaussian Markov networks using a greedy coordinate ascent approach
ECML PKDD'10 Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part III
Topology Selection in Graphical Models of Autoregressive Processes
The Journal of Machine Learning Research
Adaptive First-Order Methods for General Sparse Inverse Covariance Selection
SIAM Journal on Matrix Analysis and Applications
Solving Log-Determinant Optimization Problems by a Newton-CG Primal Proximal Point Algorithm
SIAM Journal on Optimization
Foundations and Trends® in Machine Learning
Exact covariance thresholding into connected components for large-scale graphical lasso
The Journal of Machine Learning Research
Alternating Direction Method for Covariance Selection Models
Journal of Scientific Computing
| Hi-index | 0.00 |
In this paper we first study a smooth optimization approach for solving a class of nonsmooth strictly concave maximization problems whose objective functions admit smooth convex minimization reformulations. In particular, we apply Nesterov's smooth optimization technique [Y. E. Nesterov, Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543-547; Y. E. Nesterov, Math. Programming, 103 (2005), pp. 127-152] to their dual counterparts that are smooth convex problems. It is shown that the resulting approach has ${\cal O}(1/{\sqrt{\epsilon}})$ iteration complexity for finding an $\epsilon$-optimal solution to both primal and dual problems. We then discuss the application of this approach to sparse covariance selection that is approximately solved as an $l_1$-norm penalized maximum likelihood estimation problem, and also propose a variant of this approach which has substantially outperformed the latter one in our computational experiments. We finally compare the performance of these approaches with other first-order methods, namely, Nesterov's ${\cal O}(1/\epsilon)$ smooth approximation scheme and block-coordinate descent method studied in [A. d'Aspremont, O. Banerjee, and L. El Ghaoui, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 56-66; J. Friedman, T. Hastie, and R. Tibshirani, Biostatistics, 9 (2008), pp. 432-441] for sparse covariance selection on a set of randomly generated instances. It shows that our smooth optimization approach substantially outperforms the first method above, and moreover, its variant substantially outperforms both methods above.