Learning from Examples and Membership Queries with Structured Determinations

• Authors: Prasad Tadepalli;Stuart Russell
• Venue: Machine Learning
• Year: 1998

It is well known that prior knowledge or bias can speed up learning,at least in theory. It has proved difficult to make constructive use of prior knowledge, so that approximately correcthypotheses can be learned efficiently. In this paper, we consider aparticular form of bias which consists of a set of “determinations.”A set of attributes is said to determine a given attribute ifthe latter is purely a function of the former. The bias istree-structured if there is a tree of attributes such that theattribute at any node is determined by its children, where the leavescorrespond to input attributes and the root corresponds to the targetattribute for the learning problem. The set of allowed functions ateach node is called the basis. The tree-structured biasrestricts the target functions to those representable by a read-onceformula (a Boolean formula in which each variable occurs at most once)of a given structure over the basis functions.We show that efficient learning is possible using a given tree-structured bias from random examples and membership queries, provided that the basis class itself is learnable and obeys some mild closure conditions. The algorithm uses aform of controlled experimentation in order to learn each partof the overall function, fixing the inputs to the other parts of thefunction at appropriate values. We present empirical results showing that when a tree-structured bias is available, our method significantly improves upon knowledge-free induction. We also show that there are hard cryptographic limitations to generalizing these positive results to structured determinations in the form of a directed acyclic graph.