Technical Note: \cal Q-Learning
Machine Learning
Highly dynamic Destination-Sequenced Distance-Vector routing (DSDV) for mobile computers
SIGCOMM '94 Proceedings of the conference on Communications architectures, protocols and applications
Model-based average reward reinforcement learning
Artificial Intelligence
Journal of the ACM (JACM)
Dynamic Programming and Optimal Control
Dynamic Programming and Optimal Control
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Adaptive proportional routing: a localized QoS routing approach
IEEE/ACM Transactions on Networking (TON)
Learning to Predict by the Methods of Temporal Differences
Machine Learning
An Energy-Aware QoS Routing Protocol for Wireless Sensor Networks
ICDCSW '03 Proceedings of the 23rd International Conference on Distributed Computing Systems
Mission-Critical Network Planning
Mission-Critical Network Planning
Learning to act using real-time dynamic programming
Artificial Intelligence
Autonomic wireless sensor networks
Engineering Applications of Artificial Intelligence
Capacity regions for wireless ad hoc networks
IEEE Transactions on Wireless Communications
IEEE Transactions on Multimedia
Informationally Decentralized Video Streaming Over Multihop Wireless Networks
IEEE Transactions on Multimedia
NetCamo: camouflaging network traffic for QoS-guaranteed mission critical applications
IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans
Using feedback in collaborative reinforcement learning to adaptively optimize MANET routing
IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans
Service-Oriented Sensor-Actuator Networks [Ad Hoc and Sensor Networks]
IEEE Communications Magazine
IEEE Journal on Selected Areas in Communications
Dynamic power allocation and routing for time-varying wireless networks
IEEE Journal on Selected Areas in Communications
IEEE Journal on Selected Areas in Communications
Journal of Network and Computer Applications
International Journal of Ad Hoc and Ubiquitous Computing
Hi-index | 0.07 |
In this paper, we study how to optimize the transmission decisions of nodes aimed at supporting mission-critical applications, such as surveillance, security monitoring, and military operations, etc. We focus on a network scenario where multiple source nodes transmit simultaneously mission-critical data through relay nodes to one or multiple destinations in multi-hop wireless Mission-Critical Networks (MCN). In such a network, the wireless nodes can be modeled as agents that can acquire local information from their neighbors and, based on this available information, can make timely transmission decisions to minimize the end-to-end delays of the mission-critical applications. Importantly, the MCN needs to cope in practice with the time-varying network dynamics. Hence, the agents need to make transmission decisions by considering not only the current network status, but also how the network status evolves over time, and how this is influenced by the actions taken by the nodes. We formulate the agents' autonomic decision making problem as a Markov decision process (MDP) and construct a distributed MDP framework, which takes into consideration the informationally-decentralized nature of the multi-hop MCN. We further propose an online model-based reinforcement learning approach for agents to solve the distributed MDP at runtime, by modeling the network dynamics using priority queuing. We compare the proposed model-based reinforcement learning approach with other model-free reinforcement learning approaches in the MCN. The results show that the proposed model-based reinforcement learning approach for mission-critical applications not only outperforms myopic approaches without learning capability, but also outperforms conventional model-free reinforcement learning approaches.