The nature of statistical learning theory
The nature of statistical learning theory
Upper bounds on empirically optimal quantizers
IEEE Transactions on Information Theory
| Hi-index | 0.00 |
The conditional mean is an optimal predictor in the sense of minimizing the mean square error between a random variable and a prediction using another random variable. In order to find the conditional mean, a non parametric estimator, the Nadaraya-Watson estimator with an indicator function for the kernel, is considered using m sample pairs. It is known that the estimator converges to the conditional mean in the mean of order 2 with rate m-4/5 [7]. Note that the minimized error is the expectation of the conditional variance and that the estimator minimizes the empirical error, which is an estimate of the expectation of the conditional variance. It is also known that, for the simple linear regression model and parametric estimators that have a form of the affine function, the bias of the minimized empirical error shows the rate of m-1 [4]. In this paper, for discrete distributions with the Nadaraya-Watson estimator for the predictors, the biases of the minimized empirical error and the error induced by the estimator are explicitly derived, and it is shown that the convergence rate is equal to m-1. Some discussions with examples are also shown in this paper.