Spectrum management of cognitive radio using multi-agent reinforcement learning

Authors:
Cheng Wu;Kaushik Chowdhury;Marco Di Felice;Waleed Meleis
Affiliations:
Northeastern University, Boston, MA;Northeastern University, Boston, MA;University of Bologna, Bologna, Italy;Northeastern University, Boston, MA
Venue:
Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: Industry track
Year:
2010

Citing 9
Cited 3

Principles of mobile communication (2nd ed.)

Principles of mobile communication (2nd ed.)
Sparse Distributed Memory

Sparse Distributed Memory
Brains, Behavior and Robotics

Brains, Behavior and Robotics
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Introduction to Multiagent Systems

Introduction to Multiagent Systems
NeXt generation/dynamic spectrum access/cognitive radio wireless networks: a survey

Computer Networks: The International Journal of Computer and Telecommunications Networking
CRAHNs: Cognitive radio ad hoc networks

Ad Hoc Networks
Spectrum management in cognitive radio ad hoc networks

IEEE Network: The Magazine of Global Internetworking - Special issue title on networking over multi-hop cognitive networks
Cognitive radio: brain-empowered wireless communications

IEEE Journal on Selected Areas in Communications

Learning-Based spectrum selection in cognitive radio ad hoc networks

WWIC'10 Proceedings of the 8th international conference on Wired/Wireless Internet Communications
Partially decentralized reinforcement learning in finite, multi-agent Markov decision processes

AI Communications
Performance of distributed multi-agent multi-state reinforcement spectrum management using different exploration schemes

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

Wireless cognitive radio (CR) is a newly emerging paradigm that attempts to opportunistically transmit in licensed frequencies, without affecting the pre-assigned users of these bands. To enable this functionality, such a radio must predict its operational parameters, such as transmit power and spectrum. These tasks, collectively called spectrum management, is difficult to achieve in a dynamic distributed environment, in which CR users may only take local decisions, and react to the environmental changes. In this paper, we introduce a multi-agent reinforcement learning approach based spectrum management. Our approach uses value functions to evaluate the desirability of choosing different transmission parameters, and enables efficient assignment of spectrums and transmit powers by maximizing long-term reward. We then investigate various real-world scenarios, and compare the communication performance using different sets of learning parameters. We also apply Kanerva-based function approximation to improve our approach's ability to handle large cognitive radio networks and evaluate its effect on communication performance. We conclude that our reinforcement learning based spectrum management can significantly reduce the interference to the licensed users, while maintaining a high probability of successful transmissions in a cognitive radio ad hoc network.