Cryptographic limitations on learning Boolean formulae and finite automata

Authors:
Michael Kearns;Leslie Valiant
Affiliations:
AT&T Bell Laboratories, Murray Hill, New Jersey;Harvard University, Cambridge, Massachusetts
Venue:
Journal of the ACM (JACM)
Year:
1994

Citing 26
Cited 109

A theory of the learnable

Communications of the ACM
How to generate cryptographically strong sequences of pseudo-random bits

SIAM Journal on Computing
How to construct random functions

Journal of the ACM (JACM)
Log depth circuits for division and related problems

SIAM Journal on Computing
One-way functions and pseudorandom generators

STOC '85 Proceedings of the seventeenth annual ACM symposium on Theory of computing
On the learnability of Boolean formulae

STOC '87 Proceedings of the nineteenth annual ACM symposium on Theory of computing
Occam's razor

Information Processing Letters
Learning regular sets from queries and counterexamples

Information and Computation
RSA and Rabin functions: certain parts are as hard as the whole

SIAM Journal on Computing - Special issue on cryptography
Computational limitations on learning from examples

Journal of the ACM (JACM)
The minimum consistent DFA problem cannot be approximated within and polynomial

STOC '89 Proceedings of the twenty-first annual ACM symposium on Theory of computing
An O(n0.4)-approximation algorithm for 3-coloring

STOC '89 Proceedings of the twenty-first annual ACM symposium on Theory of computing
Learnability and the Vapnik-Chervonenkis dimension

Journal of the ACM (JACM)
Learning in neural networks

COLT '88 Proceedings of the first annual workshop on Computational learning theory
Training a 3-node neural network is NP-complete

COLT '88 Proceedings of the first annual workshop on Computational learning theory
Equivalence of models for polynomial learnability

COLT '88 Proceedings of the first annual workshop on Computational learning theory
On the learnability of finite automata

COLT '88 Proceedings of the first annual workshop on Computational learning theory
A polynomial-time algorithm for learning k-variable pattern languages from examples

COLT '89 Proceedings of the second annual workshop on Computational learning theory
When won't membership queries help?

STOC '91 Proceedings of the twenty-third annual ACM symposium on Theory of computing
A method for obtaining digital signatures and public-key cryptosystems

Communications of the ACM
Computers and Intractability: A Guide to the Theory of NP-Completeness

Computers and Intractability: A Guide to the Theory of NP-Completeness
The Design and Analysis of Computer Algorithms

The Design and Analysis of Computer Algorithms
Queries and Concept Learning

Machine Learning
Queries and Concept Learning

Machine Learning
A new approximate graph coloring algorithm

STOC '82 Proceedings of the fourteenth annual ACM symposium on Theory of computing
DIGITALIZED SIGNATURES AND PUBLIC-KEY FUNCTIONS AS INTRACTABLE AS FACTORIZATION

DIGITALIZED SIGNATURES AND PUBLIC-KEY FUNCTIONS AS INTRACTABLE AS FACTORIZATION

Some weak learning results

COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
Diversity-based inference of finite automata

Journal of the ACM (JACM)
Rationality

COLT '95 Proceedings of the eighth annual conference on Computational learning theory
Exactly learning automata with small cover time

COLT '95 Proceedings of the eighth annual conference on Computational learning theory
Simple learning algorithms using divide and conquer

COLT '95 Proceedings of the eighth annual conference on Computational learning theory
Evaluation may be easier than generation (extended abstract)

STOC '96 Proceedings of the twenty-eighth annual ACM symposium on Theory of computing
On the boosting ability of top-down decision tree learning algorithms

STOC '96 Proceedings of the twenty-eighth annual ACM symposium on Theory of computing
Exactly Learning Automata of Small Cover Time

Machine Learning - Special issue on the eighth annual conference on computational learning theory, (COLT '95)
Computational sample complexity

COLT '97 Proceedings of the tenth annual conference on Computational learning theory
A dichotomy theorem for learning quantified Boolean formulas

COLT '97 Proceedings of the tenth annual conference on Computational learning theory
Structural results about exact learning with unspecified attribute values

COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
Learning from Examples and Membership Queries with Structured Determinations

Machine Learning
Computational sample complexity and attribute-efficient learning

STOC '99 Proceedings of the thirty-first annual ACM symposium on Theory of computing
A simple, fast, and effective rule learner

AAAI '99/IAAI '99 Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence
Learning to Reason with a Restricted View

Machine Learning
A Dichotomy Theorem for Learning Quantified Boolean Formulas

Machine Learning - Special issue: computational learning theory, COLT '97
Learning functions represented as multiplicity automata

Journal of the ACM (JACM)
Cryptographic limitations on parallelizing membership and equivalence queries with applications to random-self-reductions

Theoretical Computer Science - Algorithmic learning theory
Using computational learning strategies as a tool for combinatorial optimization

Annals of Mathematics and Artificial Intelligence
Learning Regular Languages from Simple Positive Examples

Machine Learning
PAC Analogues of Perceptron and Winnow Via Boosting the Margin

Machine Learning
Boosting Methods for Regression

Machine Learning
Boosting and Hard-Core Set Construction

Machine Learning
iBoost: Boosting Using an i nstance-Based Exponential Weighting Scheme

ECML '02 Proceedings of the 13th European Conference on Machine Learning
On the Learnability of Hidden Markov Models

ICGI '02 Proceedings of the 6th International Colloquium on Grammatical Inference: Algorithms and Applications
From Computational Learning Theory to Discovery Science

ICAL '99 Proceedings of the 26th International Colloquium on Automata, Languages and Programming
Separating Quantum and Classical Learning

ICALP '01 Proceedings of the 28th International Colloquium on Automata, Languages and Programming,
On the Minimal Hardware Complexity of Pseudorandom Function Generators

STACS '01 Proceedings of the 18th Annual Symposium on Theoretical Aspects of Computer Science
On Distribution-Specific Learning with Membership Queries versus Pseudorandom Generation

FST TCS 2000 Proceedings of the 20th Conference on Foundations of Software Technology and Theoretical Computer Science
Theoretical Views of Boosting

EuroCOLT '99 Proceedings of the 4th European Conference on Computational Learning Theory
Learnability of Quantified Formulas

EuroCOLT '99 Proceedings of the 4th European Conference on Computational Learning Theory
Cryptographic Limitations on Parallelizing Membership and Equivalence Queries with Applications to Random Self-Reductions

ALT '98 Proceedings of the 9th International Conference on Algorithmic Learning Theory
Theoretical Views of Boosting and Applications

ALT '99 Proceedings of the 10th International Conference on Algorithmic Learning Theory
Boolean Formulas are Hard to Learn for most Gate Bases

ALT '99 Proceedings of the 10th International Conference on Algorithmic Learning Theory
On the Hardness of Learning Acyclic Conjunctive Queries

ALT '00 Proceedings of the 11th International Conference on Algorithmic Learning Theory
Optimally-Smooth Adaptive Boosting and Application to Agnostic Learning

ALT '02 Proceedings of the 13th International Conference on Algorithmic Learning Theory
The Complexity of Learning Concept Classes with Polynomial General Dimension

ALT '02 Proceedings of the 13th International Conference on Algorithmic Learning Theory
Prediction-Preserving Reducibility with Membership Queries on Formal Languages

FCT '01 Proceedings of the 13th International Symposium on Fundamentals of Computation Theory
On Boosting with Optimal Poly-Bounded Distributions

COLT '01/EuroCOLT '01 Proceedings of the 14th Annual Conference on Computational Learning Theory and and 5th European Conference on Computational Learning Theory
Learning Regular Sets with an Incomplete Membership Oracle

COLT '01/EuroCOLT '01 Proceedings of the 14th Annual Conference on Computational Learning Theory and and 5th European Conference on Computational Learning Theory
Boosting in the presence of noise

Proceedings of the thirty-fifth annual ACM symposium on Theory of computing
Boosting and Hard-Core Sets

FOCS '99 Proceedings of the 40th Annual Symposium on Foundations of Computer Science
On the difficulty of approximately maximizing agreements

Journal of Computer and System Sciences
An introduction to boosting and leveraging

Advanced lectures on machine learning
Pseudorandom functions in TC0 and cryptographic limitations to proving lower bounds

Computational Complexity
On boosting with polynomially bounded distributions

The Journal of Machine Learning Research
Optimally-smooth adaptive boosting and application to agnostic learning

The Journal of Machine Learning Research
Smooth boosting and learning with malicious noise

The Journal of Machine Learning Research
Learnability of quantified formulas

Theoretical Computer Science
Number-theoretic constructions of efficient pseudo-random functions

Journal of the ACM (JACM)
PAC-learnability of Probabilistic Deterministic Finite State Automata

The Journal of Machine Learning Research
Theory revision with queries: horn, read-once, and parity formulas

Artificial Intelligence
On the influence of the variable ordering for algorithmic learning using OBDDs

Information and Computation
Boosting in the presence of noise

Journal of Computer and System Sciences - Special issue: Learning theory 2003
Learning a circuit by injecting values

Proceedings of the thirty-eighth annual ACM symposium on Theory of computing
The complexity of learning concept classes with polynomial general dimension

Theoretical Computer Science - Algorithmic learning theory(ALT 2002)
Prediction-hardness of acyclic conjunctive queries

Theoretical Computer Science - Algorithmic learning theory (ALT 2000)
Learning deterministic context free grammars: The Omphalos competition

Machine Learning
Interpolation of depth-3 arithmetic circuits with two multiplication gates

Proceedings of the thirty-ninth annual ACM symposium on Theory of computing
Minimizing nfa's and regular expressions

Journal of Computer and System Sciences
The complexity of properly learning simple concept classes

Journal of Computer and System Sciences
iBoost: Boosting using an instance-based exponential weighting scheme

International Journal of Hybrid Intelligent Systems
Learning large-alphabet and analog circuits with value injection queries

Machine Learning
Improving dynamic software analysis by applying grammar inference principles

Journal of Software Maintenance and Evolution: Research and Practice - Special Issue on Program Comprehension through Dynamic Analysis (PCODA)
Optimally Learning Social Networks with Activations and Suppressions

ALT '08 Proceedings of the 19th international conference on Algorithmic Learning Theory
A tamper-proof and lightweight authentication scheme

Pervasive and Mobile Computing
Evolvability

Journal of the ACM (JACM)
Learning unary automata

Journal of Automata, Languages and Combinatorics
Learning a circuit by injecting values

Journal of Computer and System Sciences
Efficient learning algorithms yield circuit lower bounds

Journal of Computer and System Sciences
Cryptographic hardness for learning intersections of halfspaces

Journal of Computer and System Sciences
Property Testing: A Learning Theory Perspective

Foundations and Trends® in Machine Learning
A PSO Based Adaboost Approach to Object Detection

SEAL '08 Proceedings of the 7th International Conference on Simulated Evolution and Learning
Knowledge infusion

AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
A brief introduction to boosting

IJCAI'99 Proceedings of the 16th international joint conference on Artificial intelligence - Volume 2
Accurate and efficient processor performance prediction via regression tree based modeling

Journal of Systems Architecture: the EUROMICRO Journal
On the influence of the variable ordering for algorithmic learning using OBDDs

Information and Computation
Learning large-alphabet and analog circuits with value injection queries

COLT'07 Proceedings of the 20th annual conference on Learning theory
An improved Adaboost.R algorithm and its application in mining safety monitoring

IITA'09 Proceedings of the 3rd international conference on Intelligent information technology application
Multi-labeled Chinese text categorization based on the boosting algorithms

ICNC'09 Proceedings of the 5th international conference on Natural computation
A fast face detection method based on improved sample selection

FSKD'09 Proceedings of the 6th international conference on Fuzzy systems and knowledge discovery - Volume 5
Public-key cryptography from different assumptions

Proceedings of the forty-second ACM symposium on Theory of computing
Optimally learning social networks with activations and suppressions

Theoretical Computer Science
Towards general algorithms for grammatical inference

ALT'10 Proceedings of the 21st international conference on Algorithmic learning theory
Arithmetic Circuits: A survey of recent results and open questions

Foundations and Trends® in Theoretical Computer Science
The Sign-Rank of AC$^0$

SIAM Journal on Computing
PCPs and the hardness of generating private synthetic data

TCC'11 Proceedings of the 8th conference on Theory of cryptography
Hierarchical Estimator

Expert Systems with Applications: An International Journal
Closing the learning-planning loop with predictive state representations

International Journal of Robotics Research
Risk estimation for hierarchical classifier

HAIS'11 Proceedings of the 6th international conference on Hybrid artificial intelligent systems - Volume Part I
Exact learning algorithms, betting games, and circuit lower bounds

ICALP'11 Proceedings of the 38th international colloquim conference on Automata, languages and programming - Volume Part I
Semantic communication for simple goals is equivalent to on-line learning

ALT'11 Proceedings of the 22nd international conference on Algorithmic learning theory
Advances in boosting

UAI'02 Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence
Restriction access

Proceedings of the 3rd Innovations in Theoretical Computer Science Conference
On optimal learning algorithms for multiplicity automata

COLT'06 Proceedings of the 19th annual conference on Learning Theory
Efficient learning algorithms yield circuit lower bounds

COLT'06 Proceedings of the 19th annual conference on Learning Theory
Goals in the propositional horn⊃ language are monotone boolean circuits

MFCS'05 Proceedings of the 30th international conference on Mathematical Foundations of Computer Science
Minimizing NFA's and regular expressions

STACS'05 Proceedings of the 22nd annual conference on Theoretical Aspects of Computer Science
A complete characterization of statistical query learning with applications to evolvability

Journal of Computer and System Sciences
Learning schema mappings

Proceedings of the 15th International Conference on Database Theory
New adaboost algorithm based on interval-valued fuzzy sets

IDEAL'12 Proceedings of the 13th international conference on Intelligent Data Engineering and Automated Learning
Evolvability

MFCS'07 Proceedings of the 32nd international conference on Mathematical Foundations of Computer Science
Regular inference as vertex coloring

ALT'12 Proceedings of the 23rd international conference on Algorithmic Learning Theory
On the learnability of shuffle ideals

ALT'12 Proceedings of the 23rd international conference on Algorithmic Learning Theory
When homomorphism becomes a liability

TCC'13 Proceedings of the 10th theory of cryptography conference on Theory of Cryptography
Automated identification of normal and diabetes heart rate signals using nonlinear measures

Computers in Biology and Medicine
Learning schema mappings

ACM Transactions on Database Systems (TODS) - Invited papers issue
Exact Learning Algorithms, Betting Games, and Circuit Lower Bounds

ACM Transactions on Computation Theory (TOCT)
On the learnability of shuffle ideals

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.01

Visualization

Abstract

In this paper, we prove the intractability of learning several classes of Boolean functions in the distribution-free model (also called the Probably Approximately Correct or PAC model) of learning from examples. These results are representation independent, in that they hold regardless of the syntactic form in which the learner chooses to represent its hypotheses.Our methods reduce the problems of cracking a number of well-known public-key cryptosystems to the learning problems. We prove that a polynomial-time learning algorithm for Boolean formulae, deterministic finite automata or constant-depth threshold circuits would have dramatic consequences for cryptography and number theory. In particular, such an algorithm could be used to break the RSA cryptosystem, factor Blum integers (composite numbers equivalent to 3 modulo 4), and detect quadratic residues. The results hold even if the learning algorithm is only required to obtain a slight advantage in prediction over random guessing. The techniques used demonstrate an interesting duality between learning and cryptography.We also apply our results to obtain strong intractability results for approximating a generalization of graph coloring.