A well-conditioned estimator for large-dimensional covariance matrices
Journal of Multivariate Analysis
The Journal of Machine Learning Research
| Hi-index | 0.00 |
High dimensional covariance estimation is an important topic in contemporary multivariate statistics and has recently received much attention in the mathematical statistics literature. The work of Bickel and Levina (2008) [2] introduces a general approach to such estimation problems in a large class of models: banding of the sample covariance matrix. Bickel and Levina show that banded estimators are consistent in the operator norm as the dimension of the covariance matrix, p, and the sample size, n, both go to infinity. Critically, these estimators rely on knowing the order of the covariates apriori before banding can be applied. A rigorous framework for order recovery is however not available in the literature. In this paper, we propose a novel framework and methodology that can be used to recover covariate order in general classes of banded models. Such models can also be framed as autoregressive processes, which in turn fall within the class of graphical models. We show that recovering covariate order is intimately related to minimizing functionals on the symmetric group. Indeed, an important contribution of the paper is a result showing that the natural time order in such an autoregressive process has the property that over all orderings of covariates, it minimizes the sum of the diagonals of the Cholesky decomposition, of both the covariance and the inverse covariance matrix. This result lays the foundation for the ensuing statistical methodology developed in this paper: an efficient algorithm called the Best Permutation Algorithm (BPA). The BPA can recover the natural order of variables in autoregressive models at the rate of O"p((logp)/n), which is the same rate that the covariance matrix can be estimated if the natural time order were known. Hence the BPA yields the oracle rate. Moreover, the computational complexity of the BPA is proved to be polynomial in the number of variables, p, and hence allows for an efficient search over the full permutation group on p letters, a group whose size is super-exponential in p. The methodology is also successfully illustrated on numerical examples.