Inductive learning in less than one sequential data scan

  • Authors:
  • Wei Fan;Haixun Wang;Philip S. Yu;Shaw-Hwa Lo

  • Affiliations:
  • IBM T.J.Watson Research, Hawthorne, NY;IBM T.J.Watson Research, Hawthorne, NY;IBM T.J.Watson Research, Hawthorne, NY;Statistics Department, Columbia University, New York, NY

  • Venue:
  • IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

Most recent research of scalable inductive learning on very large dataset, decision tree construction in particular, focuses on eliminating memory constraints and reducing the number of sequential data scans. However, state-of-the-art decision tree construction algorithms still require multiple scans over the data set and use sophisticated control mechanisms and data structures. We first discuss a general inductive learning framework that scans the dataset exactly once. Then, we propose an extension based on Hoeffding's inequality that scans the dataset less than once. Our frameworks are applicable to a wide range of inductive learners.