REDUS: finding reducible subspaces in high dimensional data
Proceedings of the 17th ACM conference on Information and knowledge management
SLICE: A Novel Method to Find Local Linear Correlations by Constructing Hyperplanes
APWeb/WAIM '09 Proceedings of the Joint International Conferences on Advances in Data and Web Management
Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery
Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
Finding multiple global linear correlations in sparse and noisy data sets
Knowledge-Based Systems
| Hi-index | 0.00 |
Finding latent patterns in high dimensional data is an important research problem with numerous applications. Existing approaches can be summarized into 3 categories: feature selection, feature transformation (or feature projection) and projected clustering. Being widely used in many applications, these methods aim to capture global patterns and are typically performed in the full feature space. In many emerging biomedical applications, however, scientists are interested in the local latent patterns held by feature subsets, which may be invisible via any global transformation. In this paper, we investigate the problem of finding local linear correlations in high dimensional data. Our goal is to find the latent pattern structures that may exist only in some subspaces. We formalize this problem as finding strongly correlated feature subsets which are supported by a large portion of the data points. Due to the combinatorial nature of the problem and lack of monotonicity of the correlation measurement, it is prohibitively expensive to exhaustively explore the whole search space. In our algorithm, CARE, we utilize spectrum properties and effective heuristic to prune the search space. Extensive experimental results show that our approach is effective in finding local linear correlations that may not be identified by existing methods.