A summary-attainment-surface plotting method for visualizing the performance of stochastic multiobjective optimizers

• Authors: Joshua Knowles
• Venue: ISDA '05 Proceedings of the 5th International Conference on Intelligent Systems Design and Applications
• Year: 2005

When evaluating the performance of a stochastic optimizer it is sometimes desirable to express peformance in terms of the quality attained in a certain fraction of sample runs. For example, the sample median quality is the best estimator of what one would expect to achieve in 50% of runs, and similarly for other quantiles. In multiobjective optimization, the notion still applies but the outcome of a run is measured not as a scalar (i.e. the cost of the best solution), but as an attainment surface in the k-dimensional space where k is the number of objects). In this paper we report an algorithm that can be conveniently used to plot summary attainment surfaces in any number of dimensions (though it is particularly suited for three). A summary attainment surface is defined as the union of all tightest goals that have been attained (independently) in precisely s of the runs of a sample of \eta runs, for any s \in 1...\eta and for any k We also discuss the computational complexity of the algorithm and give some examples of its use. C code for the algorithm is available from the author.