Elements of information theory
Elements of information theory
Unsupervised learning by probabilistic latent semantic analysis
Machine Learning
Unsupervised document classification using sequential information maximization
SIGIR '02 Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
Speeded-Up Robust Features (SURF)
Computer Vision and Image Understanding
Modeling hidden topics on document manifold
Proceedings of the 17th ACM conference on Information and knowledge management
Probabilistic dyadic data analysis with local and global consistency
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Unsupervised object category discovery via information bottleneck method
Proceedings of the international conference on Multimedia
Adapting visual category models to new domains
ECCV'10 Proceedings of the 11th European conference on Computer vision: Part IV
| Hi-index | 0.00 |
Given the joint distribution p(X,Y) of the original variable X and relevant variable Y, the Information Bottleneck (IB) method aims to extract an informative representation of the variable X by compressing it into a "bottleneck" variable T, while maximally preserving the relevant information about the variable Y. In practical applications, when the variable X is compressed into its representation T, however, this method does not take into account the local geometrical property hidden in data spaces, therefore, it is not appropriate to deal with non-linearly separable data. To solve this problem, in this study, we construct an information theoretic framework by integrating local geometrical structures into the IB methods, and propose Locally-Consistent Information Bottleneck (LCIB) method. The LCIB method uses k-nearest neighbor graph to model the local structure, and employs mutual information to measure and guarantee the local consistency of data representations. To find the optimal solution of LCIB algorithm, we adopt a sequential "draw-and-merge" procedure to achieve the converge of our proposed objective function. Experimental results on real data sets demonstrate the effectiveness of the proposed approach.