Using local density information to improve IB algorithms

Authors:
Yangdong Ye;Yongli Ren;Gang Li
Affiliations:
School of Information Engineering, Zhengzhou University, Zhengzhou 450052, China;School of Information Technology, Deakin University, 221 Burwood Highway, VIC 3125, Australia;School of Information Technology, Deakin University, 221 Burwood Highway, VIC 3125, Australia
Venue:
Pattern Recognition Letters
Year:
2011

Citing 7
Cited 0

Document clustering using word clusters via the information bottleneck method

SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Normalized Cuts and Image Segmentation

IEEE Transactions on Pattern Analysis and Machine Intelligence
Unsupervised document classification using sequential information maximization

SIGIR '02 Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
Applying the Information Bottleneck Principle to Unsupervised Clustering of Discrete and Continuous Image Representations

ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
Spectral Segmentation with Multiscale Graph Decomposition

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Visual cue cluster construction via information bottleneck principle and kernel density estimation

CIVR'05 Proceedings of the 4th international conference on Image and Video Retrieval
Unsupervised image-set clustering using an information theoretic framework

IEEE Transactions on Image Processing

Quantified Score

Hi-index	0.10

Visualization

Abstract

The Information Bottleneck principle provides a systematic method to extract relevant features from complex data sets, and it models features extraction as data compression and quantifies the relevance of extracted feature by how much information it preserved about a specified feature. How to construct an optimal solution to IB remains a problem. The current Information Bottleneck (IB) algorithms only utilize the information between element pairs, and ignore the information among the neighborhood of elements. This is one of the major reasons for most IB algorithms' failure to preserve as much relative information as possible, which further limits IB applicability in many areas. In this paper, we present the concept of density connectivity component, by which the information loss among the neighbors of an element, rather than the information loss between paired elements, can be considered. Then, we introduce this concept into the current agglomerative IB algorithm (aIB) and sequential IB algorithm (sIB), and propose two density-based IB algorithms, DaIB and DsIB. The experiment results on the benchmark data sets indicate that the DaIB and DsIB algorithm can preserve more relevant information and achieve higher precision than the aIB and sIB algorithm, respectively.