Mathematics of Operations Research
Characterization and computation of restless bandit marginal productivity indices
Proceedings of the 2nd international conference on Performance evaluation methodologies and tools
A (2/3)n3 Fast-Pivoting Algorithm for the Gittins Index and Optimal Stopping of a Markov Chain
INFORMS Journal on Computing
A Faster Index Algorithm and a Computational Study for Bandits with Switching Costs
INFORMS Journal on Computing
Characterization and computation of restless bandit marginal productivity indices
Proceedings of the 2nd international conference on Performance evaluation methodologies and tools
A Marginal Productivity Index Rule for Scheduling Multiclass Queues with Setups
Network Control and Optimization
Hi-index | 0.00 |
We address the multiarmed bandit problem with switching penalties including both costs and delays. Asawa and Teneketzis (1996) introduced an index for bandits with switching penalties that partially characterizes optimal policies, attaching to each project state a "continuation index" (its Gittins index) and a "switching index," yet only proposed an index algorithm for the case of switching costs. We present a fast decoupled computation method, which in a first stage computes the continuation index and then, in a second stage, computes the switching index an order of magnitude faster in at most (5/2)n2 + O(n) arithmetic operations for an n-state project. This extends earlier work where we introduced a two-stage index algorithm for the case of switching costs only. We exploit the fact that the Asawa and Teneketzis index is the marginal productivity index of a classic bandit with switching penalties in its semi-Markov restless reformulation, by deploying methods introduced by the author. A computational study demonstrates the dramatic runtime savings achieved by the new algorithm, the near-optimality of the index policy, and its substantial gains against the benchmark Gittins index policy across a wide range of two-and three-project instances.