Multi-conditional learning: generative/discriminative training for clustering and classification

Authors:
Andrew McCallum;Chris Pal;Greg Druck;Xuerui Wang
Affiliations:
Department of Computer Science, University of Massachusetts, Amherst, MA;Department of Computer Science, University of Massachusetts, Amherst, MA;Department of Computer Science, University of Massachusetts, Amherst, MA;Department of Computer Science, University of Massachusetts, Amherst, MA
Venue:
AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Year:
2006

Citing 11
Cited 21

Information processing in dynamical systems: foundations of harmony theory

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
Hierarchical mixtures of experts and the EM algorithm

Neural Computation
A maximum entropy approach to natural language processing

Computational Linguistics
Maximum conditional likelihood via bound maximization and the CEM algorithm

Proceedings of the 1998 conference on Advances in neural information processing systems II
Text Classification from Labeled and Unlabeled Documents using EM

Machine Learning - Special issue on information retrieval
Training products of experts by minimizing contrastive divergence

Neural Computation
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Latent dirichlet allocation

The Journal of Machine Learning Research
A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data

The Journal of Machine Learning Research
On information regularization

UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Factor graphs and the sum-product algorithm

IEEE Transactions on Information Theory

Semi-supervised classification with hybrid generative/discriminative methods

Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Generalized component analysis for text with heterogeneous attributes

Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Classification using discriminative restricted Boltzmann machines

Proceedings of the 25th international conference on Machine learning
An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators

Proceedings of the 25th international conference on Machine learning
Mining the web for visual concepts

Proceedings of the 9th International Workshop on Multimedia Data Mining: held in conjunction with the ACM SIGKDD 2008
Interpretation of hybrid generative/discriminative algorithms

Neurocomputing
MedLDA: maximum margin supervised topic models for regression and classification

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Knowledge derived from wikipedia for computing semantic relatedness

Journal of Artificial Intelligence Research
On the generative-discriminative tradeoff approach: Interpretation, asymptotic efficiency and classification performance

Computational Statistics & Data Analysis
Exponential family hybrid semi-supervised learning

IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Semi-supervised learning of visual classifiers from web images and text

IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Panel discussion

ICML'06 Proceedings of the 2006 conference on Statistical network analysis
Joint discriminative-generative modelling based on statistical tests for classification

Pattern Recognition Letters
LDA based similarity modeling for question answering

SS '10 Proceedings of the NAACL HLT 2010 Workshop on Semantic Search
Bayesian hybrid generative discriminative learning based on finite Liouville mixture models

Pattern Recognition
Domain adaptation by constraining inter-domain variability of latent feature representation

HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
Learning algorithms for the classification restricted Boltzmann machine

The Journal of Machine Learning Research
Editors Choice Article: I2VM: Incremental import vector machines

Image and Vision Computing
A hybrid semi-supervised topic model

IScIDE'11 Proceedings of the Second Sino-foreign-interchange conference on Intelligent Science and Intelligent Data Engineering
A hybrid generative/discriminative method for semi-supervised classification

Knowledge-Based Systems
A jointly distributed semi-supervised topic model

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents multi-conditional learning (MCL), a training criterion based on a product of multiple conditional likelihoods. When combining the traditional conditional probability of "label given input" with a generative probability of "input given label" the later acts as a surprisingly effective rerularizer. When applied to models with latent variables, MCL combines the structure-discovery capabilities of generative topic models, such as latent Dirichlet allocation and the exponential family harmonium, with the accuracy and robustness of discriminative classifiers, such as logistic regression and conditional random fields. We present results on several standard text data sets showing significant reductions in classification error due to MCL regularization, and substantial gains in precision and recall due to the latent structure discovered under MCL.