Computational limitations on learning from examples

Authors:
Leonard Pitt;Leslie G. Valiant
Affiliations:
Univ. of Illinois, Urbana-Champaign, Urbana;Harvard Univ., Cambridge, MA
Venue:
Journal of the ACM (JACM)
Year:
1988

Citing 13
Cited 127

A theory of the learnable

Communications of the ACM
How to construct random functions

Journal of the ACM (JACM)
Inductive inference of approximations

Information and Control
Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension

STOC '86 Proceedings of the eighteenth annual ACM symposium on Theory of computing
Machine learning an artificial intelligence approach volume II

Machine learning an artificial intelligence approach volume II
Occam's razor

Information Processing Letters
The Complexity of Near-Optimal Graph Coloring

Journal of the ACM (JACM)
Inference of Reversible Languages

Journal of the ACM (JACM)
Inductive Inference: Theory and Methods

ACM Computing Surveys (CSUR)
Computers and Intractability: A Guide to the Theory of NP-Completeness

Computers and Intractability: A Guide to the Theory of NP-Completeness
Functions Computable in the Limit by Probabilistic Machines

Proceedings of the 3rd Symposium on Mathematical Foundations of Computer Science
A new approximate graph coloring algorithm

STOC '82 Proceedings of the fourteenth annual ACM symposium on Theory of computing
A study of grammatical inference

A study of grammatical inference

Probabilistic inductive inference

Journal of the ACM (JACM)
The minimum consistent DFA problem cannot be approximated within and polynomial

STOC '89 Proceedings of the twenty-first annual ACM symposium on Theory of computing
Crytographic limitations on learning Boolean formulae and finite automata

STOC '89 Proceedings of the twenty-first annual ACM symposium on Theory of computing
Learnability and the Vapnik-Chervonenkis dimension

Journal of the ACM (JACM)
Learnable and Nonlearnable Visual Concepts

IEEE Transactions on Pattern Analysis and Machine Intelligence
On the necessity of Occam algorithms

STOC '90 Proceedings of the twenty-second annual ACM symposium on Theory of computing
Computational learning theory: survey and selected bibliography

STOC '92 Proceedings of the twenty-fourth annual ACM symposium on Theory of computing
Fast learning of k-term DNF formulas with queries

STOC '92 Proceedings of the twenty-fourth annual ACM symposium on Theory of computing
On-line learning of rectangles

COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
An O(nlog log n) learning algorithm for DNF under the uniform distribution

COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
Learning k-term DNF formulas with an incomplete membership oracle

COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
Learning DNF formulae under classes of probability distributions

COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
Toward efficient agnostic learning

COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
Robust trainability of single neurons

COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
The minimum consistent DFA problem cannot be approximated within any polynomial

Journal of the ACM (JACM)
Learning read-once formulas with queries

Journal of the ACM (JACM)
A Partially Supervised Learning Algorithm for Linearly Separable Systems

IEEE Transactions on Pattern Analysis and Machine Intelligence
Efficient learning of typical finite automata from random walks

STOC '93 Proceedings of the twenty-fifth annual ACM symposium on Theory of computing
Efficient noise-tolerant learning from statistical queries

STOC '93 Proceedings of the twenty-fifth annual ACM symposium on Theory of computing
Linear time deterministic learning of k-term DNF

COLT '93 Proceedings of the sixth annual conference on Computational learning theory
Average case analysis of the clipped Hebb rule for nonoverlapping perception networks

COLT '93 Proceedings of the sixth annual conference on Computational learning theory
Learning unions of two rectangles in the plane with equivalence queries

COLT '93 Proceedings of the sixth annual conference on Computational learning theory
Analysis of upper bound in Valiant's model for learning bounded CNF expressions

SAC '93 Proceedings of the 1993 ACM/SIGAPP symposium on Applied computing: states of the art and practice
Cryptographic limitations on learning Boolean formulae and finite automata

Journal of the ACM (JACM)
Bayesian inductive logic programming

COLT '94 Proceedings of the seventh annual conference on Computational learning theory
On the limits of proper learnability of subclasses of DNF formulas

COLT '94 Proceedings of the seventh annual conference on Computational learning theory
Oracles and queries that are sufficient for exact learning (extended abstract)

COLT '94 Proceedings of the seventh annual conference on Computational learning theory
Learning monotone log-term DNF formulas

COLT '94 Proceedings of the seventh annual conference on Computational learning theory
Learning unions of boxes with membership and equivalence queries

COLT '94 Proceedings of the seventh annual conference on Computational learning theory
The representation of recursive languages and its impact on the efficiency of learning

COLT '94 Proceedings of the seventh annual conference on Computational learning theory
Exploiting random walks for learning

COLT '94 Proceedings of the seventh annual conference on Computational learning theory
Efficient distribution-free learning of probabilistic concepts

Journal of Computer and System Sciences - Special issue: 31st IEEE conference on foundations of computer science, Oct. 22–24, 1990
Weakly learning DNF and characterizing statistical query learning using Fourier analysis

STOC '94 Proceedings of the twenty-sixth annual ACM symposium on Theory of computing
Learning Boolean formulas

Journal of the ACM (JACM)
Rationality

COLT '95 Proceedings of the eighth annual conference on Computational learning theory
Proper learning algorithm for functions of k terms under smooth distributions

COLT '95 Proceedings of the eighth annual conference on Computational learning theory
On learning bounded-width branching programs

COLT '95 Proceedings of the eighth annual conference on Computational learning theory
PAC learning axis-aligned rectangles with respect to product distributions from multiple-instance examples

COLT '96 Proceedings of the ninth annual conference on Computational learning theory
PAC learning intersections of halfspaces with membership queries (extended abstract)

COLT '96 Proceedings of the ninth annual conference on Computational learning theory
An Experimental and Theoretical Comparison of Model SelectionMethods

Machine Learning - Special issue on the eighth annual conference on computational learning theory, (COLT '95)
Approximating hyper-rectangles: learning and pseudo-random sets

STOC '97 Proceedings of the twenty-ninth annual ACM symposium on Theory of computing
Computational sample complexity

COLT '97 Proceedings of the tenth annual conference on Computational learning theory
Learning with maximum-entropy distributions

COLT '97 Proceedings of the tenth annual conference on Computational learning theory
Generating all maximal independent sets of bounded-degree hypergraphs

COLT '97 Proceedings of the tenth annual conference on Computational learning theory
Property testing and its connection to learning and approximation

Journal of the ACM (JACM)
PAC Learning Axis-aligned Rectangles with Respect toProduct Distributions from Multiple-Instance Examples

Machine Learning - Special issue on the ninth annual conference on computational theory (COLT '96)
Hardness Results for Learning First-Order Representations and Programming by Demonstration

Machine Learning - Special issue on the ninth annual conference on computational theory (COLT '96)
Identification criteria and lower bounds for perceptron-like learning rules

Neural Computation
Efficient noise-tolerant learning from statistical queries

Journal of the ACM (JACM)
An apprentice learning model (extended abstract)

COLT '99 Proceedings of the twelfth annual conference on Computational learning theory
On PAC learning using Winnow, Perceptron, and a Perceptron-like algorithm

COLT '99 Proceedings of the twelfth annual conference on Computational learning theory
Learning to Take Actions

Machine Learning
Learning binary matroid ports

SODA '94 Proceedings of the fifth annual ACM-SIAM symposium on Discrete algorithms
A Theoretical and Empirical Study of a Noise-TolerantAlgorithm to Learn Geometric Patterns

Machine Learning
A neuroidal architecture for cognitive computation

Journal of the ACM (JACM)
Complexity of learning in artificial neural networks

Theoretical Computer Science - Phase transitions in combinatorial problems
Cryptographic limitations on parallelizing membership and equivalence queries with applications to random-self-reductions

Theoretical Computer Science - Algorithmic learning theory
Learning with Maximum-Entropy Distributions

Machine Learning
Finding tree patterns consistent with positive and negative examples using queries

Annals of Mathematics and Artificial Intelligence
Molecular computing paradigm – toward freedom from Turing's charm

Natural Computing: an international journal
The complexity of minimizing and learning OBDDs and FBDDs

Discrete Applied Mathematics
Training a single sigmoidal neuron is hard

Neural Computation
Many-layered learning

Neural Computation
PAC Meditation on Boolean Formulas

Proceedings of the 5th International Symposium on Abstraction, Reformulation and Approximation
Structural Complexity and Neural Networks

WIRN VIETRI 2002 Proceedings of the 13th Italian Workshop on Neural Nets-Revised Papers
Generalized Graph Colorability and Compressibility of Boolean Formulae

ISAAC '98 Proceedings of the 9th International Symposium on Algorithms and Computation
Cryptographic Limitations on Parallelizing Membership and Equivalence Queries with Applications to Random Self-Reductions

ALT '98 Proceedings of the 9th International Conference on Algorithmic Learning Theory
Minimizing the Quadratic Training Error of a Sigmoid Neuron Is Hard

ALT '01 Proceedings of the 12th International Conference on Algorithmic Learning Theory
Learning of Boolean Functions Using Support Vector Machines

ALT '01 Proceedings of the 12th International Conference on Algorithmic Learning Theory
Solving Computational Learning Problems of Boolean Formulae on DNA Computers

DNA '00 Revised Papers from the 6th International Workshop on DNA-Based Computers: DNA Computing
Bounds for the Minimum Disagreement Problem with Applications to Learning Theory

COLT '02 Proceedings of the 15th Annual Conference on Computational Learning Theory
On the Proper Learning of Axis Parallel Concepts

COLT '02 Proceedings of the 15th Annual Conference on Computational Learning Theory
Effects of domain characteristics on instance-based learning algorithms

Theoretical Computer Science - Selected papers in honour of Setsuo Arikawa
On the difficulty of approximately maximizing agreements

Journal of Computer and System Sciences
DNA-based algorithms for learning Boolean formulae

Natural Computing: an international journal
Simplifying decision trees: A survey

The Knowledge Engineering Review
Version spaces and the consistency problem

Artificial Intelligence
Some Dichotomy Theorems for Neural Learning Problems

The Journal of Machine Learning Research
On data classification by iterative linear partitioning

Discrete Applied Mathematics - Discrete mathematics & data mining (DM & DM)
Some connections between learning and optimization

Discrete Applied Mathematics - Discrete mathematics & data mining (DM & DM)
Maximizing Agreements with One-Sided Error with Applications to Heuristic Learning

Machine Learning
On the Nonlearnability of a Single Spiking Neuron

Neural Computation
Hardness of approximate two-level logic minimization and PAC learning with membership queries

Proceedings of the thirty-eighth annual ACM symposium on Theory of computing
Revising threshold functions

Theoretical Computer Science
Rule quality for multiple-rule classifier: Empirical expertise and theoretical methodology

Intelligent Data Analysis
A general comparison of language learning from examples and from queries

Theoretical Computer Science
The complexity of properly learning simple concept classes

Journal of Computer and System Sciences
Implicit niching in a learning classifier system: Nature's way

Evolutionary Computation
On learning perceptrons with binary weights

Neural Computation
Measuring teachability using variants of the teaching dimension

Theoretical Computer Science
Parameterized Learnability of k-Juntas and Related Problems

ALT '07 Proceedings of the 18th international conference on Algorithmic Learning Theory
Learning Languages from Bounded Resources: The Case of the DFA and the Balls of Strings

ICGI '08 Proceedings of the 9th international colloquium on Grammatical Inference: Algorithms and Applications
Evolvability

Journal of the ACM (JACM)
Learning unary automata

Journal of Automata, Languages and Combinatorics
Hardness of approximate two-level logic minimization and PAC learning with membership queries

Journal of Computer and System Sciences
Efficient learning algorithms yield circuit lower bounds

Journal of Computer and System Sciences
Exact learning of random DNF over the uniform distribution

Proceedings of the forty-first annual ACM symposium on Theory of computing
Knowledge infusion

AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Pac-learning recursive logic programs: negative results

Journal of Artificial Intelligence Research
Learning from partial observations

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Parameterized learnability of juntas

Theoretical Computer Science
Logical analysis of binary data with missing bits

Artificial Intelligence
On data classification by iterative linear partitioning

Discrete Applied Mathematics
Some connections between learning and optimization

Discrete Applied Mathematics
Computational aspects of data mining

ICCSA'03 Proceedings of the 2003 international conference on Computational science and its applications: PartI
Computational science and data mining

ICCS'03 Proceedings of the 2003 international conference on Computational science: PartIII
Partial observability and learnability

Artificial Intelligence
Completing networks using observed data

ALT'09 Proceedings of the 20th international conference on Algorithmic learning theory
Negative selection algorithms without generating detectors

Proceedings of the 12th annual conference on Genetic and evolutionary computation
Learning the large-scale structure of the MAX-SAT landscape using populations

IEEE Transactions on Evolutionary Computation
Probably approximately correct learning

AAAI'90 Proceedings of the eighth National conference on Artificial intelligence - Volume 2
Learnability in inductive logic programming: some basic results and techniques

AAAI'93 Proceedings of the eleventh national conference on Artificial intelligence
Version spaces without boundary sets

AAAI'97/IAAI'97 Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence
Computational learning theory

Algorithms and theory of computation handbook
Learning random monotone DNF

Discrete Applied Mathematics
PCPs and the hardness of generating private synthetic data

TCC'11 Proceedings of the 8th conference on Theory of cryptography
Efficient learning algorithms yield circuit lower bounds

COLT'06 Proceedings of the 19th annual conference on Learning Theory
Bounds on the sample complexity for private learning and private data release

TCC'10 Proceedings of the 7th international conference on Theory of Cryptography
Learnability of bipartite ranking functions

COLT'05 Proceedings of the 18th annual conference on Learning Theory
Teaching classes with high teaching dimension using few examples

COLT'05 Proceedings of the 18th annual conference on Learning Theory
An approach to guided learning of boolean functions

Mathematical and Computer Modelling: An International Journal
From synapses to rules

Cognitive Systems Research
Activized learning: transforming passive to active with improved label complexity

The Journal of Machine Learning Research
GAMoN: Discovering M-of-N{¬,∨} hypotheses for text classification by a lattice-based Genetic Algorithm

Artificial Intelligence
Evolvability

MFCS'07 Proceedings of the 32nd international conference on Mathematical Foundations of Computer Science
Learnability of DNF with representation-specific queries

Proceedings of the 4th conference on Innovations in Theoretical Computer Science
Bounds on the sample complexity for private learning and private data release

Machine Learning

Quantified Score

Hi-index	0.02

Visualization

Abstract

The computational complexity of learning Boolean concepts from examples is investigated. It is shown for various classes of concept representations that these cannot be learned feasibly in a distribution-free sense unless R = NP. These classes include (a) disjunctions of two monomials, (b) Boolean threshold functions, and (c) Boolean formulas in which each variable occurs at most once. Relationships between learning of heuristics and finding approximate solutions to NP-hard optimization problems are given.