Sampled fictitious play for approximate dynamic programming
Computers and Operations Research
| Hi-index | 0.01 |
A Complex System can be defined as a natural, artificial, social, or economic entity whose model involves an inordinate, or even infinite number of variables. This thesis is an attempt to employ techniques from Markov Chains, Game Theory, and Functional Analysis to develop theory and algorithms for optimizing mathematical abstractions of Euclidean, discrete, and infinite dimensional complex systems. The first chapter focuses on Markov Chain techniques. (i) We develop a novel rigorous connection between the famous Small World phenomenon and effective Markov candidate generators for continuous optimization; (ii) We propose Adaptive Search with Amorphous Probabilities (ASAP), a unified algorithmic framework that allows for very general acceptance probabilities, adaptive tuning of parameters and guarantees convergence in probability to the optimum function value for continuous as well as stochastic optimization problems; (iii) We develop the notion of Fastest Mixing Boltzmann Chains to analytically formulate the "to cool or not to cool" problem for the first time. The second chapter relies on Game Theory to solve deterministic and stochastic discrete black-box optimization problems. We propose a bounded rational version of Sampled Fictitious Play (SFP). Unlike earlier work, the players in this version (i) Use samples of size one for their best response computations, (ii) Are forced to make "mistakes", (iii) Have finite memory, and (iv) Are guaranteed to find optimal solutions. In the third chapter, we study two infinite dimensional complex systems: (i) Countably infinite linear programs, and (ii) Infinite horizon production planning under convex costs. We develop duality theory for countably infinite linear programs, and a characterization of their extreme points through positive variables. We develop the Shadow Simplex method that requires finite computation in each iteration and achieves value convergence. Unlike previous research in the convex production planning domain, we allow backlogging. Properties of convex network flow problems are then used to compute closed form formulas for a minimum forecast horizon.