On online learning of decision lists

Authors:
Ziv Nevo;Ran El-Yaniv
Affiliations:
Department of Computer Science, Technion, Haifa, 32000, Israel;Department of Computer Science, Technion, Haifa, 32000, Israel
Venue:
The Journal of Machine Learning Research
Year:
2003

Citing 17
Cited 2

A theory of the learnable

Communications of the ACM
Occam's razor

Information Processing Letters
Mistake bounds and logarithmic linear-threshold learning algorithms

Mistake bounds and logarithmic linear-threshold learning algorithms
Learning Nested Differences of Intersection-Closed Concept Classes

Machine Learning
Learning functions of k terms

COLT '90 Proceedings of the third annual workshop on Computational learning theory
Computational learning theory: an introduction

Computational learning theory: an introduction
Rank-r decision trees are a subclass of r-decision lists

Information Processing Letters
Learning in the presence of finitely or infinitely many irrelevant attributes

Journal of Computer and System Sciences
Attribute-efficient learning in query and mistake-bound models

Journal of Computer and System Sciences
Computational sample complexity and attribute-efficient learning

Journal of Computer and System Sciences
Projection Learning

Machine Learning
Monotone term decision lists

Theoretical Computer Science
Learning Decision Lists

Machine Learning
Queries and Concept Learning

Machine Learning
Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm

Machine Learning
Queries and Concept Learning

Machine Learning
On-line Algorithms in Machine Learning

Developments from a June 1996 seminar on Online algorithms: the state of the art

Kernels as features: On kernels, margins, and low-dimensional mappings

Machine Learning
Toward Attribute Efficient Learning of Decision Lists and Parities

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

A fundamental open problem in computational learning theory is whether there is an attribute efficient learning algorithm for the concept class of decision lists (Rivest, 1987; Blum, 1996). We consider a weaker problem, where the concept class is restricted to decision lists with D alternations. For this class, we present a novel online algorithm that achieves a mistake bound of O(rDlog n), where r is the number of relevant variables, and n is the total number of variables. The algorithm can be viewed as a strict generalization of the famous Winnow algorithm by Littlestone (1988), and improves the O(r2Dlog n) mistake bound of Balanced Winnow. Our bound is stronger than a similar PAC-learning result of Dhagat and Hellerstein (1994). A combination of our algorithm with the algorithm suggested by Rivest (1987) might achieve even better bounds.