Linear function neurons: Structure and training
Biological Cybernetics
Learning in threshold networks
COLT '88 Proceedings of the first annual workshop on Computational learning theory
Polynomial threshold functions, AC0 functions, and spectral norms
SIAM Journal on Computing
On the Size of Weights for Threshold Gates
SIAM Journal on Discrete Mathematics
The amazing power of pairwise independence (abstract)
STOC '94 Proceedings of the twenty-sixth annual ACM symposium on Theory of computing
Discrete neural computation: a theoretical foundation
Discrete neural computation: a theoretical foundation
On specifying Boolean functions by labelled examples
Discrete Applied Mathematics
On the degree of Boolean functions as real polynomials
Computational Complexity - Special issue on circuit complexity
Perceptrons, PP, and the polynomial hierarchy
Computational Complexity - Special issue on circuit complexity
On the Fourier spectrum of monotone functions
Journal of the ACM (JACM)
Anti-Hadamard matrices, coin weighing, threshold gates, and indecomposable hypergraphs
Journal of Combinatorial Theory Series A
On connectionist models
SIAM Journal on Discrete Mathematics
ON THE HARDNESS OF APPROXIMATING MULTICUT AND SPARSEST-CUT
Computational Complexity
Every Linear Threshold Function has a Low-Weight Approximator
Computational Complexity
Learning Monotone Decision Trees in Polynomial Time
SIAM Journal on Computing
Vertex cover might be hard to approximate to within 2-ε
Journal of Computer and System Sciences
Agnostically Learning Halfspaces
SIAM Journal on Computing
Computational Complexity
The influence of variables on Boolean functions
SFCS '88 Proceedings of the 29th Annual Symposium on Foundations of Computer Science
Randomness-efficient oblivious sampling
SFCS '94 Proceedings of the 35th Annual Symposium on Foundations of Computer Science
Concentration of Measure for the Analysis of Randomized Algorithms
Concentration of Measure for the Analysis of Randomized Algorithms
Improved Lower Bounds for Embeddings into $L_1$
SIAM Journal on Computing
On Agnostic Learning of Parities, Monomials, and Halfspaces
SIAM Journal on Computing
Monotone circuits for monotone weighted threshold functions
Information Processing Letters
Learning nested halfspaces and uphill decision trees
COLT'07 Proceedings of the 20th annual conference on Learning theory
On the hardness of learning intersections of two halfspaces
Journal of Computer and System Sciences
SIAM Journal on Computing
Bounded Independence Fools Halfspaces
SIAM Journal on Computing
Explicit Construction of a Small $\epsilon$-Net for Linear Threshold Functions
SIAM Journal on Computing
SIAM Journal on Computing
Hi-index | 0.00 |
We prove two main results on how arbitrary linear threshold functions $${f(x) = {\rm sign}(w \cdot x - \theta)}$$ over the n-dimensional Boolean hypercube can be approximated by simple threshold functions. Our first result shows that every n-variable threshold function f is $${\epsilon}$$ -close to a threshold function depending only on $${{\rm Inf}(f)^2 \cdot {\rm poly}(1/\epsilon)}$$ many variables, where $${{\rm Inf}(f)}$$ denotes the total influence or average sensitivity of f. This is an exponential sharpening of Friedgut's well-known theorem (Friedgut in Combinatorica 18(1):474---483, 1998), which states that every Boolean function f is $${\epsilon}$$-close to a function depending only on $${2^{O({\rm Inf}(f)/\epsilon)}}$$ many variables, for the case of threshold functions. We complement this upper bound by showing that $${\Omega({\rm Inf}(f)^2 + 1/\epsilon^2)}$$ many variables are required for $${\epsilon}$$-approximating threshold functions. Our second result is a proof that every n-variable threshold function is $${\epsilon}$$-close to a threshold function with integer weights at most $${{\rm poly}(n) \cdot 2^{\tilde{O}(1/\epsilon^{2/3})}.}$$ This is an improvement, in the dependence on the error parameter $${\epsilon}$$, on an earlier result of Servedio (Comput Complex 16(2):180---209, 2007) which gave a $${{\rm poly}(n) \cdot 2^{\tilde{O}(1/\epsilon^{2})}}$$ bound. Our improvement is obtained via a new proof technique that uses strong anti-concentration bounds from probability theory. The new technique also gives a simple and modular proof of the original result of Servedio (Comput Complex 16(2):180---209, 2007) and extends to give low-weight approximators for threshold functions under a range of probability distributions other than the uniform distribution.