The nature of statistical learning theory
The nature of statistical learning theory
Scale-sensitive dimensions, uniform convergence, and learnability
Journal of the ACM (JACM)
Sequential and Parallel Algorithms for Mixed Packing and Covering
FOCS '01 Proceedings of the 42nd IEEE symposium on Foundations of Computer Science
Bounded Geometries, Fractals, and Low-Distortion Embeddings
FOCS '03 Proceedings of the 44th Annual IEEE Symposium on Foundations of Computer Science
Navigating nets: simple algorithms for proximity search
SODA '04 Proceedings of the fifteenth annual ACM-SIAM symposium on Discrete algorithms
Distance--Based Classification with Lipschitz Functions
The Journal of Machine Learning Research
Fast Construction of Nets in Low-Dimensional Metrics and Their Applications
SIAM Journal on Computing
Searching dynamic point sets in spaces with bounded doubling dimension
Proceedings of the thirty-eighth annual ACM symposium on Theory of computing
Cover trees for nearest neighbor
ICML '06 Proceedings of the 23rd international conference on Machine learning
Sparse solutions for linear prediction problems
Sparse solutions for linear prediction problems
All of Nonparametric Statistics (Springer Texts in Statistics)
All of Nonparametric Statistics (Springer Texts in Statistics)
Triangulation and embedding using small sets of beacons
Journal of the ACM (JACM)
Proximity algorithms for nearly-doubling spaces
APPROX/RANDOM'10 Proceedings of the 13th international conference on Approximation, and 14 the International conference on Randomization, and combinatorial optimization: algorithms and techniques
A tree-based regressor that adapts to intrinsic dimension
Journal of Computer and System Sciences
Structural risk minimization over data-dependent hierarchies
IEEE Transactions on Information Theory
Nonparametric estimation via empirical risk minimization
IEEE Transactions on Information Theory
Hi-index | 0.00 |
We present a framework for performing efficient regression in general metric spaces. Roughly speaking, our regressor predicts the value at a new point by computing a Lipschitz extension -- the smoothest function consistent with the observed data -- while performing an optimized structural risk minimization to avoid overfitting. The offline (learning) and online (inference) stages can be solved by convex programming, but this naive approach has runtime complexity O(n3), which is prohibitive for large datasets. We design instead an algorithm that is fast when the doubling dimension, which measures the "intrinsic" dimensionality of the metric space, is low. We make dual use of the doubling dimension: first, on the statistical front, to bound fat-shattering dimension of the class of Lipschitz functions (and obtain risk bounds); and second, on the computational front, to quickly compute a hypothesis function and a prediction based on Lipschitz extension. Our resulting regressor is both asymptotically strongly consistent and comes with finite-sample risk bounds, while making minimal structural and noise assumptions.