Adaption of stepsize parameter using newton's method

Authors:
Itsuki Noda
Affiliations:
AIST, Tsukuba Univ. and Tokyo Inst. of Tech., Tsukuba, Japan
Venue:
PRIMA'11 Proceedings of the 14th international conference on Agents in Principle, Agents in Practice
Year:
2011

Citing 5
Cited 0

Multiagent learning using a variable learning rate

Artificial Intelligence
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Learning Rates for Q-learning

The Journal of Machine Learning Research
Adaptive stepsizes for recursive estimation with applications in approximate dynamic programming

Machine Learning
Recursive adaptation of stepsize parameter for non-stationary environments

ALA'09 Proceedings of the Second international conference on Adaptive and Learning Agents

Quantified Score

Hi-index	0.00

Visualization

Abstract

A method to optimize stepsize parameters in exponential moving average (EMA) based on Newton's method to minimize square errors is proposed. The stepsize parameters used in reinforcement learning methods should be selected and adjusted carefully for dynamic and non-stationary environments. To find the suitable values for the stepsize parameters through learning, a framework to acquire higher-order derivatives of learning values by the stepsize parameters has been proposed. Based on this framework, the authors extend a method to determine the best stepsize using Newton's method to minimize EMA of square error of learning. The method is confirmed by mathematical theories and by results of experiments.