Lower bounds for hit-and-run direct search

• Authors: Jens Jägersküpper
• Venue: SAGA'07 Proceedings of the 4th international conference on Stochastic Algorithms: foundations and applications
• Year: 2007

"Hit-and-run is fast and fun" to generate a random point in a high dimensional convex set K (Lovász/Vempala, MSR-TR-2003-05). More precisely, the hit-and-run random walk mixes fast independently of where it is started inside the convex set. To hit-and-run from a point x ∈ Rn, a line L through x is randomly chosen (uniformly over all directions). Subsequently, the walk's next point is sampled from L ∩ K using a membership oracle which tells us whether a point is in K or not. Here the focus is on black-box optimization, however, where the function f : Rn → R to be minimized is given as an oracle, namely a black box for f-evaluations. We obtain in an obvious way a direct-search method when we substitute the f-oracle for the K-membership oracle to do a line search over L, and, naturally, we are interested in how fast such a hit-and-run direct-search heuristic converges to the optimum point x* in the search space Rn. We prove that, even under the assumption of perfect line search, the search converges (at best) linearly at an expected rate larger (i. e. worse) than 1-1/n. This implies a lower bound of 0.5 n on the expected number of line searches necessary to halve the approximation error. Moreover, we show that 0.4 n line searches suffice to halve the approximation error only with an exponentially small probability of exp(-Ω(n1/3)). Since each line search requires at least one query to the f-oracle, the lower bounds obtained hold also for the number of f-evaluations.