Fast curvature matrix-vector products for second-order gradient descent
Neural Computation
| Hi-index | 0.00 |
Stochastic meta-descent (SMD) is a new technique for online adaptation of local learning rates in arbitrary twice-differentiable systems. Like matrix momentum it uses full second-order information while retaining O(n) computational complexity by exploiting the efficient computation of Hessian-vector products. Here we apply SMD to independent component analysis, and employ the resulting algorithm for the blind separation of time-varying mixtures. By matching individual learning rates to the rate of change in each source signal''s mixture coefficients, our technique is capable of simultaneously tracking sources that move at very different, a priori unknown speeds.