Support vector machine learning for interdependent and structured output spaces

  • Authors:
  • Tomas Hofmann;Ioannis Tsochantaridis

  • Affiliations:
  • Brown University;Brown University

  • Venue:
  • Support vector machine learning for interdependent and structured output spaces
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Learning general functional dependencies between arbitrary input and output spaces is one of the main goals in machine learning. Recent progress in kernel-based methods has focused on designing flexible and powerful input representations, far example, involving sequences, strings, or graphs. This dissertation addresses the complementary issue of designing classification algorithms that can deal with more complex outputs. In particular, the problems we consider fall into two generic cases: first, problems where output themselves can be characterized by certain output-specific attributes, and learning should occur across outputs as much as across inputs; second, problems where the outputs are structured, that is, they describe a configuration over components, with possible dependencies among the components. We approach these problems by generalizing multiclass support vector machines in a maximum-margin formulation that involves features extracted jointly from inputs and outputs, and supports the use of general loss functions. This leads to a quadratic program with a potentially prohibitive, for instance, exponential, number of constraints. Nevertheless, we present a cutting plane algorithm that solves the optimization problem in polynomial time for a large class of problems. Besides the respective theoretical results, we present experiments from various domains involving different types of output spaces, that emphasize the breadth and generality of our approach. The proposed method has important applications in areas such as computational biology, natural language processing, information retrieval extraction, and optical character recognition.