COLT '90 Proceedings of the third annual workshop on Computational learning theory
The weighted majority algorithm
Information and Computation
Exponentiated gradient versus gradient descent for linear predictors
Information and Computation
Journal of the ACM (JACM)
A game of prediction with expert advice
Journal of Computer and System Sciences - Special issue on the eighth annual workshop on computational learning theory, July 5–8, 1995
Comparison of convex combination and affine combination of adaptive filters
ICASSP '09 Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing
Improving the Tracking Capability of Adaptive Filters via Convex Combination
IEEE Transactions on Signal Processing - Part II
Universal linear prediction by model order weighting
IEEE Transactions on Signal Processing
Universal Piecewise Linear Prediction Via Context Trees
IEEE Transactions on Signal Processing - Part II
Mean-square performance of a convex combination of two adaptive filters
IEEE Transactions on Signal Processing
An Affine Combination of Two LMS Adaptive Filters—Transient Mean-Square Analysis
IEEE Transactions on Signal Processing
Universal Switching Linear Least Squares Prediction
IEEE Transactions on Signal Processing
Universal portfolios with side information
IEEE Transactions on Information Theory
The context-tree weighting method: basic properties
IEEE Transactions on Information Theory
Worst-case quadratic loss bounds for prediction using linear functions and gradient descent
IEEE Transactions on Neural Networks
Adaptive mixture methods based on Bregman divergences
Digital Signal Processing
Hi-index | 35.68 |
In this paper, we consider mixture approaches that adaptively combine outputs of several parallel ruuning adaptive algorithms. These parallel units can be considered as diversity branches that can be exploited to improve the overall performance. We study various mixture structures where the final output is constructed as the weighted linear combination of the outputs of several constituent filters. Although the mixture structure is linear, the combination weights can be updated in a highly nonlinear manner to minimize the final estimation error such as in Singer and Feder 1999; Arenas-Garcia, Figueiras-Vidal, and Sayed 2006; Lopes, Satorius, and Sayed 2006; Bershad, Bermudez, and Tourneret 2008; and Silva and Nascimento 2008. We distinguish mixture approaches that are convex combinations (where the linear mixture weights are constrained to be non-negative and sum up to one) [Singer and Feder 1999; Arenas-Garcia, Figueiras-Vidal, and Sayed 2006], affine combinations (where the linear mixture weights are constrained to sum up to one) [Bershad, Bermudez, and Tourneret 2008] and, finally, unconstrained linear combinations of constituent filters [Kozat and Singer 2000]. We investigate mixture structures with respect to their final mean-square error (MSE) and tracking performance in the steady state for stationary and certain nonstationary data, respectively. We demonstrate that these mixture approaches can greatly improve over the performance of the constituent filters. Our analysis is also generic such that it can be applied to inhomogeneous mixtures of constituent adaptive branches with possibly different structures, adaptation methods or having different filter lengths.