Feature selection with complexity measure in a quadratic programming setting
IbPRIA'11 Proceedings of the 5th Iberian conference on Pattern recognition and image analysis
| Hi-index | 0.00 |
Measures of statistical dependence such as the correlation coefficient and mutual information have been widely used in variable selection. The use of correlation has been inspired by the concept of regression whereas the use of mutual information has been largely motivated by information theory. In a statistical sense, however, the concept of dependence is much broader, and extends beyond correlation and mutual information. In this paper, we explore the fundamental notion of statistical dependence in the context of variable selection. In particular, we discuss the properties of dependence as proposed by Renyi, and evaluate their significance in the variable selection context. We, also, explore a measure of dependence that satisfies most of these desired properties, and discuss its applicability as a substitute for correlation coefficient and mutual information. Finally, we compare these measures of dependence to select important variables for regression with real world data.