Exponentiated gradient versus gradient descent for linear predictors
Information and Computation
A Comparison of New and Old Algorithms for a Mixture EstimationProblem
Machine Learning - Special issue on the eighth annual conference on computational learning theory, (COLT '95)
Journal of the ACM (JACM)
A game of prediction with expert advice
Journal of Computer and System Sciences - Special issue on the eighth annual workshop on computational learning theory, July 5–8, 1995
Adaptive algorithms for sparse echo cancellation
Signal Processing
Plant identification via adaptive combination of transversal filters
Signal Processing - Signal processing in UWB communications
A class of stochastic gradient algorithms with exponentiated error cost functions
Digital Signal Processing
Steady-state MSE performance analysis of mixture approaches to adaptive filtering
IEEE Transactions on Signal Processing
Transient and steady-state analysis of the affine combination of two adaptive filters
IEEE Transactions on Signal Processing
An adaptive approach for the identification of improper complex signals
Signal Processing
Mean-square performance of a convex combination of two adaptive filters
IEEE Transactions on Signal Processing
An Affine Combination of Two LMS Adaptive Filters—Transient Mean-Square Analysis
IEEE Transactions on Signal Processing
Transient Analysis of Adaptive Affine Combinations
IEEE Transactions on Signal Processing
| Hi-index | 0.00 |
We investigate adaptive mixture methods that linearly combine outputs of m constituent filters running in parallel to model a desired signal. We use Bregman divergences and obtain certain multiplicative updates to train the linear combination weights under an affine constraint or without any constraints. We use unnormalized relative entropy and relative entropy to define two different Bregman divergences that produce an unnormalized exponentiated gradient update and a normalized exponentiated gradient update on the mixture weights, respectively. We then carry out the mean and the mean-square transient analysis of these adaptive algorithms when they are used to combine outputs of m constituent filters. We illustrate the accuracy of our results and demonstrate the effectiveness of these updates for sparse mixture systems.