Stochastic differential equations (3rd ed.): an introduction with applications
Stochastic differential equations (3rd ed.): an introduction with applications
An introduction to variational methods for graphical models
Learning in graphical models
Context-specific multiagent coordination and planning with factored MDPs
Eighteenth national conference on Artificial intelligence
Transition-independent decentralized markov decision processes
AAMAS '03 Proceedings of the second international joint conference on Autonomous agents and multiagent systems
Information Theory, Inference & Learning Algorithms
Information Theory, Inference & Learning Algorithms
Planning, learning and coordination in multiagent decision processes
TARK '96 Proceedings of the 6th conference on Theoretical aspects of rationality and knowledge
Conjugate Points in Formation Constrained Optimal Multi-Agent Coordination: A Case Study
SIAM Journal on Control and Optimization
Optimal on-line scheduling in stochastic multiagent systems in continuous space-time
Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems
Formal models and algorithms for decentralized decision making under uncertainty
Autonomous Agents and Multi-Agent Systems
Solving transition independent decentralized Markov decision processes
Journal of Artificial Intelligence Research
Approximate inference and constrained optimization
UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Factor graphs and the sum-product algorithm
IEEE Transactions on Information Theory
EP for Efficient Stochastic Control with Obstacles
Proceedings of the 2010 conference on ECAI 2010: 19th European Conference on Artificial Intelligence
A Generalized Path Integral Control Approach to Reinforcement Learning
The Journal of Machine Learning Research
Learning variable impedance control
International Journal of Robotics Research
Hi-index | 0.00 |
In this article we consider the issue of optimal control in collaborative multi-agent systems with stochastic dynamics. The agents have a joint task in which they have to reach a number of target states. The dynamics of the agents contains additive control and additive noise, and the autonomous part factorizes over the agents. Full observation of the global state is assumed. The goal is to minimize the accumulated joint cost, which consists of integrated instantaneous costs and a joint end cost. The joint end cost expresses the joint task of the agents. The instantaneous costs are quadratic in the control and factorize over the agents. The optimal control is given as a weighted linear combination of single-agent to single-target controls. The single-agent to single-target controls are expressed in terms of diffusion processes. These controls, when not closed form expressions, are formulated in terms of path integrals, which are calculated approximately by Metropolis-Hastings sampling. The weights in the control are interpreted as marginals of a joint distribution over agent to target assignments. The structure of the latter is represented by a graphical model, and the marginals are obtained by graphical model inference. Exact inference of the graphical model will break down in large systems, and so approximate inference methods are needed. We use naive mean field approximation and belief propagation to approximate the optimal control in systems with linear dynamics. We compare the approximate inference methods with the exact solution, and we show that they can accurately compute the optimal control. Finally, we demonstrate the control method in multi-agent systems with nonlinear dynamics consisting of up to 80 agents that have to reach an equal number of target states.