Communications of the ACM
Specification and simulation of statistical query algorithms for efficiency and noise tolerance
COLT '95 Proceedings of the eighth annual conference on Computational learning theory
Neural Networks for Pattern Recognition
Neural Networks for Pattern Recognition
On Learning Correlated Boolean Functions Using Statistical Queries
ALT '01 Proceedings of the 12th International Conference on Algorithmic Learning Theory
On the Efficiency of Noise-Tolerant PAC Algorithms Derived from Statistical Queries
COLT '00 Proceedings of the Thirteenth Annual Conference on Computational Learning Theory
Unconditional lower bounds for learning intersections of halfspaces
Machine Learning
Evolvability from learning algorithms
STOC '08 Proceedings of the fortieth annual ACM symposium on Theory of computing
Computational Complexity
Exploiting Product Distributions to Identify Relevant Variables of Correlation Immune Functions
The Journal of Machine Learning Research
A characterization of strong learnability in the statistical query model
STACS'07 Proceedings of the 24th annual conference on Theoretical aspects of computer science
SIAM Journal on Computing
A complete characterization of statistical query learning with applications to evolvability
Journal of Computer and System Sciences
Statistical algorithms and a lower bound for detecting planted cliques
Proceedings of the forty-fifth annual ACM symposium on Theory of computing
| Hi-index | 0.00 |
We prove two lower bounds in the statistical query (SQ) learning model. The first lower bound is on weak-learning. We prove that for a concept class of SQ-dimension d, a running time of @W(d/logd) is needed. The SQ-dimension of a concept class is defined to be the maximum number of concepts that are ''uniformly correlated'', in that each of their pair has nearly the same correlation. This lower bound matches the upper bound in Blum et al. (Weakly Learning DNF and Characterizing Statistical Query Learning using Fourier Analysis, STOC 1994, pp. 253-262), up to a logarithmic factor. We prove this lower bound against an ''honest SQ-oracle'', which gives a stronger result than the ones against the more frequently used ''adversarial SQ-oracles''. The second lower bound is more general. It gives a continuous trade-off between the ''advantage'' of an algorithm in learning the target function and the number of queries it needs to make, where the advantage of an algorithm is the probability it succeeds in predicting a label minus the probability it does not. Both lower bounds extend and/or strengthen previous results, and solve an open problem left in previous papers. An earlier version of this paper [K. Yang, New lower bounds for statistical query learning, in: The Proceedings of the 15th Annual Conference on Computational Learning Theory, COLT 2002, Sydney, Australia, July 8-10, Lecture Notes in Computer Science, vol. 2375, 2002, pp. 229-243.] appeared in the proceedings of the 15th Annual Conference on Computational Learning Theory (COLT 2002).