Covariance Matrix Estimation with Multi-Regularization Parameters based on MDL Principle

Authors:
Xiuling Zhou;Ping Guo;C. L. Chen
Affiliations:
The Laboratory of Image Processing and Pattern Recognition, Beijing Normal University, Beijing, China 100875 and Research Department, Beijing City University, Beijing, China;The Laboratory of Image Processing and Pattern Recognition, Beijing Normal University, Beijing, China 100875;The Faculty of Science & Technology, University of Macau, Macau, SAR China
Venue:
Neural Processing Letters
Year:
2013

Citing 9
Cited 0

Covariance Matrix Estimation and Classification With Limited Training Data

IEEE Transactions on Pattern Analysis and Machine Intelligence
On cross validation for model selection

Neural Computation
A well-conditioned estimator for large-dimensional covariance matrices

Journal of Multivariate Analysis
Bayesian Quadratic Discriminant Analysis

The Journal of Machine Learning Research
A study of regularized Gaussian classifier in high-dimension small sample set case based on MDL principle with application to spectrum recognition

Pattern Recognition
Noniterative map reconstruction using sparse matrix representations

IEEE Transactions on Image Processing
Paper: Modeling by shortest data description

Automatica (Journal of IFAC)
The minimum description length principle in coding and modeling

IEEE Transactions on Information Theory
The Sparse Matrix Transform for Covariance Estimation and Analysis of High Dimensional Signals

IEEE Transactions on Image Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Regularization is a solution for the problem of unstable estimation of covariance matrix with a small sample set in Gaussian classifier. In many applications such as image restoration, sparse representation, we have to deal with multi-regularization parameters problem. In this paper, the case of covariance matrix estimation with multi-regularization parameters is investigated, and an estimate method called as KLIM_L is derived theoretically based on Minimum Description Length (MDL) principle for the small sample size problem with high dimension setting. KLIM_L estimator can be regarded as a generalization of KLIM estimator in which local difference in each dimension is considered. Under the framework of MDL principle, a selection method of multi-regularization parameters is also developed based on the minimization of the Kullback-Leibler information measure, which is simply and directly estimated by point estimation under the approximation of two-order Taylor expansion. The computational cost to estimate multi-regularization parameters with KLIM_L method is less than those with RDA (Regularized Discriminant Analysis) and LOOC (leave-one-out covariance matrix estimate) in which cross validation technique is adopted. Experiments show that higher classification accuracy can be achieved by using the proposed KLIM_L estimator.