Analysis and parameter selection for an adaptive random search algorithm

• Authors: Rajeeva Kumar;David C. Hyland;Pierre T. Kabamba
• Venue: Analysis and parameter selection for an adaptive random search algorithm
• Year: 2004

This dissertation is motivated by the problem of finding a global minimizer or a feasible argument for a function of several variables. Such a problem occurs in many engineering and science disciplines. There are several challenges associated with these problems. For example, the very definition of global minimum of a function makes the verification of a candidate an NP-hard problem. Further, the cost functions of the associated optimization problem may have numerous local minima. Furthermore, in many cases, the computational requirements for computing the global minimum increase multifold with the increase in the parameter dimension. One approach that could overcome these problems is random search optimization. However, very little is known about how different parameters of such algorithms should be adjusted in order to achieve an optimal convergence speed. Also, there is hardly any practical stopping criterion for such algorithms. In this dissertation, we analyzed an Adaptive Random Search Algorithm (ARS) by developing a probability model for the number of iterations required to find a minimizer. The analysis reveals the key parameters affecting the convergence rate and provides insight on ways to tune the algorithm for optimal convergence. A new stopping criterion is also proposed for this algorithm that eliminates the need to estimate the optimum function value beforehand. This then leads to the Monte Carlo Version of the algorithm where, given a pre-specified failure probability, one can estimate the maximum number of samples needed to find an acceptable solution before hand. The Monte Carlo Version of the ARS Algorithm thus obtained, is then applied to solve two optimization problems. Using this algorithm, first we solve a multi-modal optimization problem that arises in designing a formation flight controller and demonstrate that the algorithm yields better results when compared to the Sequential Quadratic Programming method implemented in the Matlab Optimization Toolbox. In the second application, we use the ARS algorithm for solving the constrained multi-objective optimization problem of finding a combined fuel and time Pareto-optimal relative trajectory for two spacecraft in a space borne interferometry imaging system. This application shows that the ARS algorithm can be adapted easily to solve such constraint multi criteria optimization problems.