Bayesian hybrid generative discriminative learning based on finite Liouville mixture models

Authors:
Nizar Bouguila
Affiliations:
Concordia Institute for Information Systems Engineering, Faculty of Engineering and Computer Science, Concordia University, Montreal, Qc, Canada H3G 2W1
Venue:
Pattern Recognition
Year:
2011

Citing 46
Cited 5

The statistical analysis of compositional data

The statistical analysis of compositional data
Multivariate Liouville distributions

Journal of Multivariate Analysis
Fundamentals of statistical exponential families: with applications in statistical decision theory

Fundamentals of statistical exponential families: with applications in statistical decision theory
Bayesian diagnosis in expert systems

Artificial Intelligence
Texture analysis

Handbook of pattern recognition & computer vision
The nature of statistical learning theory

The nature of statistical learning theory
Neural networks for pattern recognition

Neural networks for pattern recognition
Learning and Updating of Uncertainty in Dirichlet Models

Machine Learning
The connection between regularization operators and support vector kernels

Neural Networks
Exploiting generative models in discriminative classifiers

Proceedings of the 1998 conference on Advances in neural information processing systems II
Batch and on-line parameter estimation of Gaussian mixtures based on the joint entropy

Proceedings of the 1998 conference on Advances in neural information processing systems II
Unsupervised learning by probabilistic latent semantic analysis

Machine Learning
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond

Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope

International Journal of Computer Vision
Inference in model-based cluster analysis

Statistics and Computing
Unsupervised Learning of Models for Recognition

ECCV '00 Proceedings of the 6th European Conference on Computer Vision-Part I
The Effect of the Input Density Distribution on Kernel-based Classifiers

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
An Efficient Image Similarity Measure Based on Approximations of KL-Divergence Between Two Gaussian Mixtures

ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
Distinctive Image Features from Scale-Invariant Keypoints

International Journal of Computer Vision
Feature Space Interpretation of SVMs with Indefinite Kernels

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Discriminative Framework for Modelling Object Classes

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1 - Volume 01
Discriminative Training for Object Recognition Using Image Patches

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
A Bayesian Hierarchical Model for Learning Natural Scene Categories

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
A Generative/Discriminative Learning Algorithm for Image Classification

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision - Volume 2
Supervised clustering with support vector machines

ICML '05 Proceedings of the 22nd international conference on Machine learning
Using unsupervised learning of a finite Dirichlet mixture model to improve pattern recognition applications

Pattern Recognition Letters - Special issue: Artificial neural networks in pattern recognition
Practical Bayesian estimation of a finite beta mixture through gibbs sampling and its applications

Statistics and Computing
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
Unsupervised Selection of a Finite Dirichlet Mixture Model: An MML-Based Approach

IEEE Transactions on Knowledge and Data Engineering
Continuous Optimization based-on Boosting Gaussian Mixture Mod

ICPR '06 Proceedings of the 18th International Conference on Pattern Recognition - Volume 01
Unsupervised learning of a finite discrete mixture: Applications to texture modeling and image databases summarization

Journal of Visual Communication and Image Representation
Semi-supervised classification with hybrid generative/discriminative methods

Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Probabilistic distance measures of the Dirichlet and Beta distributions

Pattern Recognition
Scene Classification Using a Hybrid Generative/Discriminative Approach

IEEE Transactions on Pattern Analysis and Machine Intelligence
Hybrid Generative-Discriminative Visual Categorization

International Journal of Computer Vision
Generative models for similarity-based classification

Pattern Recognition
Bayesian Core: A Practical Approach to Computational Bayesian Statistics

Bayesian Core: A Practical Approach to Computational Bayesian Statistics
On Bayesian analysis of a finite generalized Dirichlet mixture via a Metropolis-within-Gibbs sampling

Pattern Analysis & Applications
Learning mixture models with support vector machines for sequence classification and segmentation

Pattern Recognition
Multi-conditional learning: generative/discriminative training for clustering and classification

AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Object classification by fusing SVMs and Gaussian mixtures

Pattern Recognition
Novel mixtures based on the dirichlet distribution: application to data and image classification

MLDM'03 Proceedings of the 3rd international conference on Machine learning and data mining in pattern recognition
Segmenting dynamic textures with ising descriptors, ARX models and level sets

WDV'05/WDV'06/ICCV'05/ECCV'06 Proceedings of the 2005/2006 international conference on Dynamical vision
Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images

IEEE Transactions on Pattern Analysis and Machine Intelligence
Unsupervised learning of a finite mixture model based on the Dirichlet distribution and its application

IEEE Transactions on Image Processing
A comparison of methods for multiclass support vector machines

IEEE Transactions on Neural Networks

A Liouville-based approach for discrete data categorization

RSFDGrC'11 Proceedings of the 13th international conference on Rough sets, fuzzy sets, data mining and granular computing
Spatial color image segmentation based on finite non-Gaussian mixture models

Expert Systems with Applications: An International Journal
Generalized Gaussian mixture models as a nonparametric Bayesian approach for clustering using class-specific visual features

Journal of Visual Communication and Image Representation
On online high-dimensional spherical data clustering and feature selection

Engineering Applications of Artificial Intelligence
Expectation propagation learning of finite Beta-Liouville mixtures for spatio-temporal object recognition

Proceedings of the 2nd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Communication

Quantified Score

Hi-index	0.02

Visualization

Abstract

Recently hybrid generative discriminative approaches have emerged as an efficient knowledge representation and data classification engine. However, little attention has been devoted to the modeling and classification of non-Gaussian and especially proportional vectors. Our main goal, in this paper, is to discover the true structure of this kind of data by building probabilistic kernels from generative mixture models based on Liouville family, from which we develop the Beta-Liouville distribution, and which includes the well-known Dirichlet as a special case. The Beta-Liouville has a more general covariance structure than the Dirichlet which makes it more practical and useful. Our learning technique is based on a principled purely Bayesian approach which resulted models are used to generate support vector machine (SVM) probabilistic kernels based on information divergence. In particular, we show the existence of closed-form expressions of the Kullback-Leibler and Renyi divergences between two Beta-Liouville distributions and then between two Dirichlet distributions as a special case. Through extensive simulations and a number of experiments involving synthetic data, visual scenes and texture images classification, we demonstrate the effectiveness of the proposed approaches.