Numerical methods for stochastic control problems in continuous time
Numerical methods for stochastic control problems in continuous time
A computational description of the organization of human reaching and prehension
A computational description of the organization of human reaching and prehension
State-feedback control of systems with multiplicative noise via linear matrix inequalities
Systems & Control Letters
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
Neuro-Dynamic Programming
Cosine tuning minimizes motor errors
Neural Computation
Neural network learning of optimal Kalman prediction and control
Neural Networks
Neural learning of Kalman filtering, Kalman control, and system identification
IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
A Generalized Path Integral Control Approach to Reinforcement Learning
The Journal of Machine Learning Research
International Journal of Robotics Research
Computers in Biology and Medicine
Correlations in state space can cause sub-optimal adaptation of optimal feedback control models
Journal of Computational Neuroscience
Journal of Computational Neuroscience
| Hi-index | 0.00 |
Optimality principles of biological movement are conceptually appealing and straightforward to formulate. Testing them empirically, however, requires the solution to stochastic optimal control and estimation problems for reasonably realistic models of the motor task and the sensorimotor periphery. Recent studies have highlighted the importance of incorporating biologically plausible noise into such models. Here we extend the linear-quadratic-gaussian framework—currently the only framework where such problems can be solved efficiently—to include controldependent, state-dependent, and internal noise. Under this extended noise model, we derive a coordinate-descent algorithm guaranteed to converge to a feedback control law and a nonadaptive linear estimator optimal with respect to each other. Numerical simulations indicate that convergence is exponential, local minima do not exist, and the restriction to nonadaptive linear estimators has negligible effects in the control problems of interest. The application of the algorithm is illustrated in the context of reaching movements. A Matlab implementation is available at www.cogsci.ucsd.edu/~todorov.