DReL: a middleware for wireless sensor networks management using reinforcement learning techniques

• Authors: Kunal Shah;Mohan Kumar
• Venue: Proceedings of the 5th International Workshop on Middleware Tools, Services and Run-Time Support for Sensor Networks
• Year: 2010

Support for autonomous and adaptive management is essential for many wireless sensor network (WSN) applications expected to be functioning over long periods of time. WSN management is further complicated by heterogeneity in terms of resources as well as applications deployed on those resources. In this paper, we present, Distributed Reinforcement Learning (DReL), middleware that provides adaptive WSN management by applying techniques from reinforcement learning and utility theory. DReL exploits a two-tier learning scheme consisting of: a) micro-learner-managing node's local tasks and resources by learning utilities of performing various tasks in different states; and b) macro-learner- managing macroscopic view and actions of a node by ensuring system as a whole achieves application's goal. Novel contributions of DReL include design and development of utility based mechanisms and associated data-structures for task, data and reward distribution by adopting concepts from directed diffusion. Our work demonstrates that individual node level as well as global level learning can help in designing a generic scheme to optimize the system while maintaining robust, localized and distributed sensing as provided by directed diffusion. Through preliminary performance analysis it is shown that DReL results in substantial increase in system lifetime compared to traditional directed diffusion.