Learning structured prediction models: a large margin approach

• Authors: Daphne Koller;Ben Taskar
• Venue: Learning structured prediction models: a large margin approach
• Year: 2005

This thesis presents a novel statistical estimation framework for structured models based on the large margin principle underlying support vector machines. We consider standard probabilistic models, such as Markov networks (undirected graphical models) and context free grammars as well as less conventional combinatorial models such as weighted graph-cuts and matchings. Our framework results in several efficient learning formulations for complex prediction tasks. Fundamentally, we rely on the expressive power of convex optimization problems to compactly capture inference or solution optimality in structured models. Directly embedding this structure within the learning formulation produces compact convex problems for efficient estimation of very complex and diverse models. For some of these models, alternative estimation methods are intractable. We analyze the theoretical generalization properties of our approach and derive a novel margin-based bound for structured prediction. In order to scale up to very large training datasets, we develop problem-specific optimization algorithms that exploit efficient dynamic programming subroutines. We describe experimental applications to a diverse range of tasks, including handwriting recognition, 3D terrain classification, disulfide connectivity prediction, hypertext categorization, natural language parsing, email organization and image segmentation. These empirical evaluations show significant improvements over state-of-the-art methods and promise wide practical use for our framework.