Technical Note: \cal Q-Learning
Machine Learning
Discrete-time battery models for system-level low-power design
IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Neuro-Dynamic Programming
Energy management for battery-powered embedded systems
ACM Transactions on Embedded Computing Systems (TECS)
B#: A Battery Emulator and Power-Profiling Instrument
IEEE Design & Test
Approximate Dynamic Programming: Solving the Curses of Dimensionality (Wiley Series in Probability and Statistics)
Leakage-aware energy synchronization for wireless sensor networks
Proceedings of the 7th international conference on Mobile systems, applications, and services
Design of a solar-harvesting circuit for batteryless embedded systems
IEEE Transactions on Circuits and Systems Part I: Regular Papers
Hierarchical hybrid power supply networks
Proceedings of the 47th Design Automation Conference
Hybrid electrical energy storage systems
Proceedings of the 16th ACM/IEEE international symposium on Low power electronics and design
LiveLab: measuring wireless networks and smartphone users in the field
ACM SIGMETRICS Performance Evaluation Review
| Hi-index | 0.00 |
The operability of a portable embedded system is severely constrained by its supply's duration. We propose a novel energy management strategy for a combined (hybrid) supply consisting of a battery and a set of supercapacitors to extend the system's lifetime. Batteries are not sufficient for handling high load fluctuations and demands in modern complex systems. Supercapacitors hold promise for complementing battery supplies because they possess higher power density, a larger number of charge/recharge cycles, and less sensitivity to operational conditions. However, supercapacitors are not efficient as a stand-alone supply because of their comparatively higher leakage and lower energy density. Due to the nonlinearity of the hybrid supply elements, multiplicity of the possible supply states, and the stochastic nature of the workloads, deriving an optimal management policy is a challenge. We pose this problem as a stochastic Markov Decision Process (MDP) and develop a reinforcement learning method, called Q-learning, to derive an efficient approximation for the optimal management strategy. This method studies a diverse set of workload profiles for a mobile platform and learns the best policy in form of an adaptive approximation approach. Evaluations on measurements collected from mobile phone users show the effectiveness of our proposed method in maximizing the combined energy system's lifetime.