Sharing information for Q-learning-based network bandwidth estimation and network failure detection

• Authors: Partha S. Dutta;Nicholas R. Jennings;R. Jennings;Luc Moreau
• Venue: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems
• Year: 2005

Distributed bandwidth (bw) estimation for effective routing and network failure detection are essential in several networked applications. However, they are hard problems since the individual nodes (agents) do not have global knowledge about the network states. Network failures further change network states which becomes an additional challenge for the nodes to estimate. In this paper, to estimate bw with network failures and to detect failures, we extend an existing state of the art algorithm (delayed information (DI) sharing) [1] that uses Q-learning for bw estimation (our new algorithm is termed e-DI). Specifically, in DI, an agent distributes its local state (available bw) knowledge to the others along a call path only after a successful call connection. However, e-DI distributes state knowledge after a call failure, caused due to network congestion or node failure, in addition to a call success. This additional information aids the agents in assessing the network state changes and also in diagnosing failures. Thus, e-DI advances the state of the art in the following ways: (i) more effective bw estimation in dynamic conditions due to failures and (ii) the capability to detect failures. Empirical analysis on a simulated radio network with stop type failures show (1) e-DI attains a higher call success rate (up to 15%) than DI, (2) after failures, the call success rate of e-DI recovers faster (up to 3 times) than DI, and (3) e-DI is guaranteed both to diagnose failure only when it occurs (no false positives) and whenever it occurs (no false negatives). There is a tradeoff in that e-DI achieves these advantages at the expense of larger messages (almost 3 times) than DI.