Universal ε-approximators for integrals

• Authors: Michael Langberg;Leonard J. Schulman
• Venue: SODA '10 Proceedings of the twenty-first annual ACM-SIAM symposium on Discrete Algorithms
• Year: 2010

Let X be a space and F a family of 0, 1-valued functions on X. Vapnik and Chervonenkis showed that if F is "simple" (finite VC dimension), then for every probability measure μ on X and ε 0 there is a finite set S such that for all f ε F, Σxεs f(x)/|S| = |f f(x)dμ(x)] ± ε. Think of S as a "universal ε-approximator" for integration in F. S can actually be obtained w.h.p. just by sampling a few points from μ. This is a mainstay of computational learning theory. It was later extended by other authors to families of bounded (e.g., [0, 1]-valued) real functions. In this work we establish similar "universal ε-approximators" for families of unbounded nonnegative real functions --- in particular, for the families over which one optimizes when performing data classification. (In this case the ε-approximation should be multiplicative.) Specifically, let F be the family of "k-median functions" (or k-means, etc.) on Rd with an arbitrary norm ϱ. That is, any set u1,..., uk ε Rd determines an f by f(x) = (mini ϱ(x - ui))α. (Here α ≥ 0.) Then for every measure μ on Rd there exists a set S of cardinality poly(k, d, 1/ε) and a measure v supported on S such that for every f ε F, Σxεs f(x)v(x) ε (1 ± ε) · (f f (x)dμ(x)).