Linear function neurons: Structure and training
Biological Cybernetics
Learning in threshold networks
COLT '88 Proceedings of the first annual workshop on Computational learning theory
Surveys in combinatorics, 1993
On the Size of Weights for Threshold Gates
SIAM Journal on Discrete Mathematics
How fast can a threshold gate learn?
Proceedings of a workshop on Computational learning theory and natural learning systems (vol. 1) : constraints and prospects: constraints and prospects
Anti-Hadamard matrices, coin weighing, threshold gates, and indecomposable hypergraphs
Journal of Combinatorial Theory Series A
Neural Networks and Complexity Theory
MFCS '92 Proceedings of the 17th International Symposium on Mathematical Foundations of Computer Science
On Small Depth Threshold Circuits
SWAT '92 Proceedings of the Third Scandinavian Workshop on Algorithm Theory
On Small Depth Threshold Circuits
SWAT '92 Proceedings of the Third Scandinavian Workshop on Algorithm Theory
On connectionist models
Every Linear Threshold Function has a Low-Weight Approximator
Computational Complexity
Improved Approximation of Linear Threshold Functions
CCC '09 Proceedings of the 2009 24th Annual IEEE Conference on Computational Complexity
Nearly optimal solutions for the chow parameters problem and low-weight approximation of halfspaces
STOC '12 Proceedings of the forty-fourth annual ACM symposium on Theory of computing
Faster private release of marginals on small databases
Proceedings of the 5th conference on Innovations in theoretical computer science
| Hi-index | 0.00 |
For S ⊆ {0,1}n, a Boolean function f: S - {-1,1} is a halfspace over S if there exist w ∈ Rn and θ ∈ R such that f(x)=sign(w ⋅ x - θ) for all x ∈ S. We give bounds on the size of integer weights w1,...,wn ∈ Z that are required to represent halfspaces over Hamming balls centered at 0n, i.e. halfspaces over S ={0,1}n≤ k = {x ∈ {0,1}n : x1 + ⋅⋅⋅ + xn ≤ k}. Such weight bounds for halfspaces over Hamming balls have immediate consequences for the performance of learning algorithms in the increasingly common scenario of learning from very high-dimensional categorical examples which are such that only a small number of features are active in each example. We give upper and lower bounds on weight both for exact representation (when sign(w ⋅ x -θ) must equal f(x) for every x ∈ S) and for ε-approximate representation (when sign(w ⋅ x-θ) may disagree with f(x) for up to an ε fraction of points x ∈ S). Our results show that extremal bounds for exact representation are qualitatively rather similar whether the domain is all of {0,1}n or the Hamming ball {0,1}n≤ k, but extremal bounds for approximate representation are qualitatively very different between these two domains.