Purple SOX extraction management system

  • Authors:
  • Philip Bohannon;Srujana Merugu;Cong Yu;Vipul Agarwal;Pedro DeRose;Arun Iyer;Ankur Jain;Vinay Kakade;Mridul Muralidharan;Raghu Ramakrishnan;Warren Shen

  • Affiliations:
  • Yahoo! Research;Yahoo! Research;Yahoo! Research;Yahoo! Research;University of Wisconsin Madison;Yahoo! Research;Yahoo! Research;Yahoo! Research;Yahoo! Research;Yahoo! Research;Yahoo! Research

  • Venue:
  • ACM SIGMOD Record
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

We describe the Purple SOX (PSOX) EMS, a prototype Extraction Management System currently being built at Yahoo!. The goal of the PSOX EMS is to manage a large number of sophisticated extraction pipelines across different application domains, at the web scale and with minimum human involvement. Three key value propositions are described: extensibility, the ability to swap in and out extraction operators; explainability, the ability to track the provenance of extraction results; and social feedback support, the facility for gathering and reconciling multiple, potentially conflicting sources.