Learning DNF in time

Authors:
Adam R. Klivans;Rocco Servedio
Affiliations:
Laboratory for Computer Science, MIT, Cambridge, MA;Division of Engineering and Applied Sciences, Harvard University, Cambridge, MA
Venue:
STOC '01 Proceedings of the thirty-third annual ACM symposium on Theory of computing
Year:
2001

Citing 30
Cited 27

A theory of the learnable

Communications of the ACM
Learning decision trees from random examples needed for learning

Information and Computation
A general lower bound on the number of examples needed for learning

Information and Computation
Learnability and the Vapnik-Chervonenkis dimension

Journal of the ACM (JACM)
Learning Nested Differences of Intersection-Closed Concept Classes

Machine Learning
Learning DNF under the uniform distribution in quasi-polynomial time

COLT '90 Proceedings of the third annual workshop on Computational learning theory
The expressive power of voting polynomials

STOC '91 Proceedings of the twenty-third annual ACM symposium on Theory of computing
Perceptrons

Neurocomputing: foundations of research
Learning monotone ku DNF formulas on product distributions

COLT '91 Proceedings of the fourth annual workshop on Computational learning theory
Rank-r decision trees are a subclass of r-decision lists

Information Processing Letters
On learning visual concepts and DNF formulae

COLT '93 Proceedings of the sixth annual conference on Computational learning theory
On using the Fourier transform to learn Disjoint DNF

Information Processing Letters
On learning monotone DNF formulae under uniform distributions

Information and Computation
How fast can a threshold gate learn?

Proceedings of a workshop on Computational learning theory and natural learning systems (vol. 1) : constraints and prospects: constraints and prospects
Weakly learning DNF and characterizing statistical query learning using Fourier analysis

STOC '94 Proceedings of the twenty-sixth annual ACM symposium on Theory of computing
Fast learning of k-term DNF formulas with queries

Journal of Computer and System Sciences - Special issue on selected papers presented at the 24th annual ACM symposium on the theory of computing (STOC '92)
Boosting a weak learning algorithm by majority

Information and Computation
The perceptron algorithm vs. Winnow: linear vs. logarithmic mistake bounds when few input variables are relevant

COLT '95 Proceedings of the eighth annual conference on Computational learning theory
A subexponential exact learning algorithm for DNF using equivalence queries

Information Processing Letters
An efficient membership-query algorithm for learning DNF with respect to the uniform distribution

Journal of Computer and System Sciences
Large margin classification using the perceptron algorithm

COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
Identification criteria and lower bounds for perceptron-like learning rules

Neural Computation
Worst-case analysis of the perceptron and exponentiated update algorithms

Artificial Intelligence
On PAC learning using Winnow, Perceptron, and a Perceptron-like algorithm

COLT '99 Proceedings of the twelfth annual conference on Computational learning theory
Queries and Concept Learning

Machine Learning
Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm

Machine Learning
Queries and Concept Learning

Machine Learning
Learning DNF by Approximating Inclusion-Exclusion Formulae

COCO '99 Proceedings of the Fourteenth Annual IEEE Conference on Computational Complexity
Learning noisy perceptrons by a perceptron in polynomial time

FOCS '97 Proceedings of the 38th Annual Symposium on Foundations of Computer Science
A polynomial-time algorithm for learning noisy linear threshold functions

FOCS '96 Proceedings of the 37th Annual Symposium on Foundations of Computer Science

Exact learning of DNF formulas using DNF hypotheses

STOC '02 Proceedings of the thiry-fourth annual ACM symposium on Theory of computing
Learnability beyond AC0

STOC '02 Proceedings of the thiry-fourth annual ACM symposium on Theory of computing
Boosting and Hard-Core Set Construction

Machine Learning
On Learning Embedded Midbit Functions

ALT '02 Proceedings of the 13th International Conference on Algorithmic Learning Theory
On the Proper Learning of Axis Parallel Concepts

COLT '02 Proceedings of the 15th Annual Conference on Computational Learning Theory
New degree bounds for polynomial threshold functions

Proceedings of the thirty-fifth annual ACM symposium on Theory of computing
Complexity in the case against accuracy estimation

Theoretical Computer Science
On the proper learning of axis-parallel concepts

The Journal of Machine Learning Research
Learning intersections and thresholds of halfspaces

Journal of Computer and System Sciences - Special issue on FOCS 2002
Exact learning of DNF formulas using DNF hypotheses

Journal of Computer and System Sciences - Special issue on COLT 2002
Learning DNF from random walks

Journal of Computer and System Sciences - Special issue: Learning theory 2003
Hardness of approximate two-level logic minimization and PAC learning with membership queries

Proceedings of the thirty-eighth annual ACM symposium on Theory of computing
On learning embedded midbit functions

Theoretical Computer Science - Algorithmic learning theory(ALT 2002)
Separating AC0 from depth-2 majority circuits

Proceedings of the thirty-ninth annual ACM symposium on Theory of computing
Every Linear Threshold Function has a Low-Weight Approximator

Computational Complexity
The complexity of properly learning simple concept classes

Journal of Computer and System Sciences
Learning intersections of halfspaces with a margin

Journal of Computer and System Sciences
Extremal properties of polynomial threshold functions

Journal of Computer and System Sciences
Polynomials that Sign Represent Parity and Descartes' Rule of Signs

Computational Complexity
A lower bound for agnostically learning disjunctions

COLT'07 Proceedings of the 20th annual conference on Learning theory
Approximating multilinear monomial coefficients and maximum multilinear monomials in multivariate polynomials

COCOA'10 Proceedings of the 4th international conference on Combinatorial optimization and applications - Volume Part I
The complexity of testing monomials in multivariate polynomials

COCOA'11 Proceedings of the 5th international conference on Combinatorial optimization and applications
Algorithms for testing monomials in multivariate polynomials

COCOA'11 Proceedings of the 5th international conference on Combinatorial optimization and applications
Learning hurdles for sleeping experts

Proceedings of the 3rd Innovations in Theoretical Computer Science Conference
Learnability of DNF with representation-specific queries

Proceedings of the 4th conference on Innovations in Theoretical Computer Science
Approximating multilinear monomial coefficients and maximum multilinear monomials in multivariate polynomials

Journal of Combinatorial Optimization
On testing monomials in multivariate polynomials

Theoretical Computer Science

Quantified Score

Hi-index	0.01

Visualization

Abstract

Using techniques from learning theory, we show that any s-term DNF over n variables can be computed by a polynomial threshold function of degree O(n^{1/3} \log s). This upper bound matches, up to a logarithmic factor, the longstanding lower bound given by Minsky and Papert in their 1968 book {\em Perceptrons}. As a consequence of this upper bound we obtain the fastest known algorithm for learning polynomial size DNF, one of the central problems in computational learning theory.