Data-Driven Constructive Induction

• Authors: Eric Bloedorn;Ryszard S. Michalski
• Venue: IEEE Intelligent Systems
• Year: 1998

Inductive-learning algorithms are powerful tools for identifying meaningful patterns in large volumes of data, and their use is increasing in fields such as data mining and computer vision. However, conventional inductive-learning algorithms are selectiveýthey rely on existing, user-provided data to build their descriptions. Thus, data analysts must assume the important and sizeable task of determining relevant attributes. If they provide inadequate attributes for describing the training examples, the descriptions the program creates are likely to be excessively complex and inaccurate. Attributes can be inadequate for the learning task when they are weakly or indirectly relevant, conditionally relevant, or inappropriately measured. Constructive induction is a general approach for coping with inadequate attributes found in original data. It uses two intertwined searchesýone for the best representation space, the other for the best hypothesis within that spaceýto formulate a generalized description of examples. Originally, constructive induction focused on improving the representation space by generating additional task-relevant attributes. It was subsequently observed that this was only one way of modifying the space. Attribute construction is a form of representation space expansion; attribute selection and attribute value abstraction are forms of representation space destruction. Furthermore, it became clear that this improvement of the representation space by expansion and destruction could have a profound impact on the simplicity and predictive accuracy of concepts induced from that space. The better the representation space, the easier it is for the program to learn. It is thus important to not only add relevant attributes, but also to remove irrelevant ones and find a useful level of precision for the attribute values. Constructive induction methods are classified according to the information used to search for the best representation space: ý data-driven constructive induction (DCI) uses input examples, ý hypothesis-driven constructive induction (HCI) uses intermediate hypotheses, and ý knowledge-driven constructive induction (KCI) uses domain knowledge provided by an expert.In multistrategy constructive induction (MCI), two or more of these methods are used. This expanded definition of constructive induction guided our development of several constructive induction programs: AQ17-DCI, AQ17-HCI, and AQ17-MCI. These all use an AQ-type rule learning algorithm for conducting hypothesis search, hence the "AQ" prefix. Here we describe our latest methodology for the data-driven constructive induction, implemented in AQ17-DCI. Our methodology combines the AQ-15c learning algorithm with a range of operators for improving the representation space. These operators are classified into constructors and destructors. Constructors extend the representation space using attribute generation methods and destructors reduce the space using attribution selection methods and attribute abstraction. We integrated these operatorsýwhich are usually considered separatelyýinto AQ17-DCI in a synergistic fashion. We tested the method on two real-world problems: text categorization and natural scene interpretation. The power of a constructive induction approach is illustrated by an example from the "second Monk's problem" which was used in an international competition of machine-learning programs.