Discriminative unsupervised learning of structured predictors

  • Authors:
  • Linli Xu;Dana Wilkinson;Finnegan Southey;Dale Schuurmans

  • Affiliations:
  • University of Waterloo, Waterloo ON, Canada;University of Waterloo, Waterloo ON, Canada;University of Alberta, Edmonton AB, Canada;University of Alberta, Edmonton AB, Canada

  • Venue:
  • ICML '06 Proceedings of the 23rd international conference on Machine learning
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present a new unsupervised algorithm for training structured predictors that is discriminative, convex, and avoids the use of EM. The idea is to formulate an unsupervised version of structured learning methods, such as maximum margin Markov networks, that can be trained via semidefinite programming. The result is a discriminative training criterion for structured predictors (like hidden Markov models) that remains unsupervised and does not create local minima. To reduce training cost, we reformulate the training procedure to mitigate the dependence on semidefinite programming, and finally propose a heuristic procedure that avoids semidefinite programming entirely. Experimental results show that the convex discriminative procedure can produce better conditional models than conventional Baum-Welch (EM) training.