Semi-supervised classification with hybrid generative/discriminative methods

Authors:
Gregory Druck;Chris Pal;Andrew McCallum;Xiaojin Zhu
Affiliations:
University of Massachusetts: Amherst;University of Massachusetts: Amherst;University of Massachusetts: Amherst;University of Wisconsin-Madison
Venue:
Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Year:
2007

Citing 11
Cited 20

Multitask Learning

Machine Learning - Special issue on inductive transfer
Combining labeled and unlabeled data with co-training

COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
Learning to classify text from labeled and unlabeled documents

AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Transductive Inference for Text Classification using Support Vector Machines

ICML '99 Proceedings of the Sixteenth International Conference on Machine Learning
A comparison of algorithms for maximum entropy parameter estimation

COLING-02 proceedings of the 6th conference on Natural language learning - Volume 20
Efficient co-regularised least squares regression

ICML '06 Proceedings of the 23rd international conference on Machine learning
Principled Hybrids of Generative and Discriminative Models

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1
A Generative-Discriminative Hybrid Method for Multi-View Object Detection

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
Combining Generative and Discriminative Methods for Pixel Classification with Multi-Conditional Learning

ICPR '06 Proceedings of the 18th International Conference on Pattern Recognition - Volume 02
Large Scale Transductive SVMs

The Journal of Machine Learning Research
Multi-conditional learning: generative/discriminative training for clustering and classification

AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1

Classification using discriminative restricted Boltzmann machines

Proceedings of the 25th international conference on Machine learning
Mining the web for visual concepts

Proceedings of the 9th International Workshop on Multimedia Data Mining: held in conjunction with the ACM SIGKDD 2008
On semi-supervised learning of Gaussian mixture models for phonetic classification

SemiSupLearn '09 Proceedings of the NAACL HLT 2009 Workshop on Semi-Supervised Learning for Natural Language Processing
On the generative-discriminative tradeoff approach: Interpretation, asymptotic efficiency and classification performance

Computational Statistics & Data Analysis
Exponential family hybrid semi-supervised learning

IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Semi-supervised learning of visual classifiers from web images and text

IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Supervised self-taught learning: actively transferring knowledge from unlabeled data

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Help-training semi-supervised LS-SVM

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Generalized Expectation Criteria for Semi-Supervised Learning with Weakly Labeled Data

The Journal of Machine Learning Research
Language models learning for domain-specific natural language user interaction

ROBIO'09 Proceedings of the 2009 international conference on Robotics and biomimetics
A robust semi-supervised classification method for transfer learning

CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
Bayesian hybrid generative discriminative learning based on finite Liouville mixture models

Pattern Recognition
Help-Training for semi-supervised support vector machines

Pattern Recognition
Kernels for longitudinal data with variable sequence length and sampling intervals

Neural Computation
Learning algorithms for the classification restricted Boltzmann machine

The Journal of Machine Learning Research
A geometric view of conjugate priors

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three
A hybrid generative/discriminative method for semi-supervised classification

Knowledge-Based Systems
Inter-training: Exploiting unlabeled data in multi-classifier systems

Knowledge-Based Systems
3D segmentation of abdominal CT imagery with graphical models, conditional random fields and learning

Machine Vision and Applications
A jointly distributed semi-supervised topic model

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

We compare two recently proposed frameworks for combining generative and discriminative probabilistic classifiers and apply them to semi-supervised classification. In both cases we explore the tradeoff between maximizing a discriminative likelihood of labeled data and a generative likelihood of labeled and unlabeled data. While prominent semi-supervised learning methods assume low density regions between classes or are subject to generative modeling assumptions, we conjecture that hybrid generative/discriminative methods allow semi-supervised learning in the presence of strongly overlapping classes and reduce the risk of modeling structure in the unlabeled data that is irrelevant for the specific classification task of interest. We apply both hybrid approaches within naively structured Markov random field models and provide a thorough empirical comparison with two well-known semi-supervised learning methods on six text classification tasks. A semi-supervised hybrid generative/discriminative method provides the best accuracy in 75% of the experiments, and the multi-conditional learning hybrid approach achieves the highest overall mean accuracy across all tasks.