Review: Reinforcement learning for context awareness and intelligence in wireless networks: Review, new features and open issues

Authors:
Kok-Lim Alvin Yau;Peter Komisarczuk;Paul D. Teal
Affiliations:
Network Engineering Research Group, School of Engineering and Computer Science, Victoria University of Wellington, P.O. Box 600, Wellington 6140, New Zealand;Network Engineering Research Group, School of Engineering and Computer Science, Victoria University of Wellington, P.O. Box 600, Wellington 6140, New Zealand and Networks Research Group, School of ...;Network Engineering Research Group, School of Engineering and Computer Science, Victoria University of Wellington, P.O. Box 600, Wellington 6140, New Zealand
Venue:
Journal of Network and Computer Applications
Year:
2012

Citing 34
Cited 5

Analyzing the Short-Term Fairness of IEEE 802.11 in Wireless Multi-Hop Radio Networks

MASCOTS '02 Proceedings of the 10th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems
Efficient QoS Provisioning for Adaptive Multimedia in Mobile Communication Networks by Reinforcement Learning

BROADNETS '04 Proceedings of the First International Conference on Broadband Networks
Learning-Enforced Time Domain Routing to Mobile Sinks in Wireless Sensor Fields

LCN '04 Proceedings of the 29th Annual IEEE International Conference on Local Computer Networks
SAMPLE: Statistical Network Link Modelling in an On-Demand Probabilistic Routing Protocol for Ad Hoc Networks

WONS '05 Proceedings of the Second Annual Conference on Wireless On-demand Network Systems and Services
Mobilized Ad-Hoc Networks: A Reinforcement Learning Approach

ICAC '04 Proceedings of the First International Conference on Autonomic Computing
LQ-Routing Protocol for Mobile Ad-Hoc Networks

Proceedings of the Fourth Annual ACIS International Conference on Computer and Information Science
Heuristic and Distributed QoS Route Discovery for Mobile Ad hoc Networks

CIT '05 Proceedings of the The Fifth International Conference on Computer and Information Technology
Constrained Flooding: A Robust and Efficient Routing Framework for Wireless Sensor Networks

AINA '06 Proceedings of the 20th International Conference on Advanced Information Networking and Applications - Volume 01
Collaborative Multiagent Reinforcement Learning by Payoff Propagation

The Journal of Machine Learning Research
SARA: Stochastic Automata Rate Adaptation for IEEE 802.11 Networks

IEEE Transactions on Parallel and Distributed Systems
A Mobile-Directory Approach to Service Discovery in Wireless Ad Hoc Networks

IEEE Transactions on Mobile Computing
Can Routing Oscillations be Good? The Benefits of Route-switching in Self-aware Networks

MASCOTS '07 Proceedings of the 2007 15th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems
Towards autonomous data ferry route design through reinforcement learning

WOWMOM '08 Proceedings of the 2008 International Symposium on a World of Wireless, Mobile and Multimedia Networks
CLIQUE: Role-Free Clustering with Q-Learning for Wireless Sensor Networks

ICDCS '09 Proceedings of the 2009 29th IEEE International Conference on Distributed Computing Systems
RL-Based Queue Management for QoS Support in Multi-channel Multi-radio Mesh Networks

NCA '09 Proceedings of the 2009 Eighth IEEE International Symposium on Network Computing and Applications
Medical QoS provision based on reinforcement learning in ultrasound streaming over 3.5G wireless systems

IEEE Journal on Selected Areas in Communications - Special issue on wireless and pervasive communications for healthcare
Direct Reinforcement Learning for Autonomous Power Configuration and Control in Wireless Networks

AHS '09 Proceedings of the 2009 NASA/ESA Conference on Adaptive Hardware and Systems
A novel framework for dynamic spectrum management in multicell OFDMA networks based on reinforcement learning

WCNC'09 Proceedings of the 2009 IEEE conference on Wireless Communications & Networking Conference
QELAR: A Machine-Learning-Based Adaptive Routing Protocol for Energy-Efficient and Lifetime-Extended Underwater Sensor Networks

IEEE Transactions on Mobile Computing
Cooperative communications with relay selection for QoS provisioning in wireless sensor networks

GLOBECOM'09 Proceedings of the 28th IEEE conference on Global telecommunications
Online learning in autonomic multi-hop wireless networks for transmitting mission-critical applications

IEEE Journal on Selected Areas in Communications
Decentralized Q-learning for aggregated interference control in completely and partially observable cognitive radio networks

CCNC'10 Proceedings of the 7th IEEE conference on Consumer communications and networking conference
A Stable Online Algorithm for Energy-Efficient Multiuser Scheduling

IEEE Transactions on Mobile Computing
An adaptive link layer for heterogeneous multi-radio mobile sensor networks

IEEE Journal on Selected Areas in Communications - Special issue on simple wireless sensor networking solutions
From cognition to docition: The teaching radio paradigm for distributed & autonomous deployments

Computer Communications
Dynamic routing optimization based on real time Adaptive Delay Estimation for wireless networks

ISCC '10 Proceedings of the The IEEE symposium on Computers and Communications
Enhancing network performance in Distributed Cognitive Radio Networks using single-agent and multi-agent Reinforcement Learning

LCN '10 Proceedings of the 2010 IEEE 35th Conference on Local Computer Networks
Optimal admission control in multimedia mobile networks with handover prediction

IEEE Wireless Communications
A learning approach for prioritized handoff channel allocation in mobile multimedia networks

IEEE Transactions on Wireless Communications
A fuzzy reinforcement learning approach to power control in wireless transmitters

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Using feedback in collaborative reinforcement learning to adaptively optimize MANET routing

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans
Performance analysis of the IEEE 802.11 distributed coordination function

IEEE Journal on Selected Areas in Communications
Near-optimal reinforcement learning framework for energy-aware sensor communications

IEEE Journal on Selected Areas in Communications
Non-Cooperative Power Control for Wireless Ad Hoc Networks with Repeated Games

IEEE Journal on Selected Areas in Communications

Review: Cognitive radio network security: A survey

Journal of Network and Computer Applications
Routing in Distributed Cognitive Radio Networks: A Survey

Wireless Personal Communications: An International Journal
Reinforcement learning in hierarchical cognitive radio wireless networks

Proceedings of the 2nd ACM workshop on High performance mobile opportunistic systems
Reinforcement Learning for Multiple Access Control in Wireless Sensor Networks: Review, Model, and Open Issues

Wireless Personal Communications: An International Journal
RLAM: A dynamic and efficient reinforcement learning-based adaptive mapping scheme in mobile WiMAX networks

Mobile Information Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

In wireless networks, context awareness and intelligence are capabilities that enable each host to observe, learn, and respond to its complex and dynamic operating environment in an efficient manner. These capabilities contrast with traditional approaches where each host adheres to a predefined set of rules, and responds accordingly. In recent years, context awareness and intelligence have gained tremendous popularity due to the substantial network-wide performance enhancement they have to offer. In this article, we advocate the use of reinforcement learning (RL) to achieve context awareness and intelligence. The RL approach has been applied in a variety of schemes such as routing, resource management and dynamic channel selection in wireless networks. Examples of wireless networks are mobile ad hoc networks, wireless sensor networks, cellular networks and cognitive radio networks. This article presents an overview of classical RL and three extensions, including events, rules and agent interaction and coordination, to wireless networks. We discuss how several wireless network schemes have been approached using RL to provide network performance enhancement, and also open issues associated with this approach. Throughout the paper, discussions are presented in a tutorial manner, and are related to existing work in order to establish a foundation for further research in this field, specifically, for the improvement of the RL approach in the context of wireless networking, for the improvement of the RL approach through the use of the extensions in existing schemes, as well as for the design and implementation of RL in new schemes.