Adaptive signal processing
Linear function neurons: Structure and training
Biological Cybernetics
Advances in neural information processing systems 2
Acceleration Techniques for the Backpropagation Algorithm
Proceedings of the EURASIP Workshop 1990 on Neural Networks
Concept acquisition through representational adjustment
Concept acquisition through representational adjustment
Layered concept-learning and dynamically variable bias management
IJCAI'87 Proceedings of the 10th international joint conference on Artificial intelligence - Volume 1
Confidence-weighted linear classification for text categorization
The Journal of Machine Learning Research
Applying the learning rate adaptation to the matrix factorization based collaborative filtering
Knowledge-Based Systems
Adaptive regularization of weight vectors
Machine Learning
Reinforcement learning algorithms with function approximation: Recent advances and applications
Information Sciences: an International Journal
Hi-index | 0.00 |
Appropriate bias is widely viewed as the key to efficient learning and generalization. I present a new algorithm, the Incremental Delta-Bar-Delta (IDBD) algorithm, for the learning of appropriate biases based on previous learning experience. The IDBD algorithm is developed for the case of a simple, linear learning system--the LMS or delta rule with a separate learning-rate parameter for each input. The IDBD algorithm adjusts the learning-rate parameters, which are an important form of bias for this system. Because bias in this approach is adapted based on previous learning experience, the appropriate test beds are drifting or non-stationary learning tasks. For particular tasks of this type, I show that the IDBD algorithm performs better than ordinary LMS and in fact finds the optimal learning rates. The IDBD algorithm extends and improves over prior work by Jacobs and by me in that it is fully incremental and has only a single free parameter. This paper also extends previous work by presenting a derivation of the IDBD algorithm as gradient descent in the space of learning-rate parameters. Finally, I offer a novel interpretation of the IDBD algorithm as an incremental form of hold-one-out cross validation.