A New Class of Alternating Proximal Minimization Algorithms with Costs-to-Move

• Authors: H. Attouch;P. Redont;A. Soubeyran
• Venue: SIAM Journal on Optimization
• Year: 2007

Given two objective functions $f:\mathcal X\mapsto\R\cup\{+\infty\}$ and $g:\mathcal Y\mapsto\R\cup\{+\infty\}$ on abstract spaces $\mathcal X$ and $\mathcal Y$, and a coupling function $c:\mathcal X\times\mathcal Y\mapsto\R^+$, we introduce and study alternative minimization algorithms of the following type: $(x_0,y_0)\in\mathcal X\times\mathcal Y \mbox{ given; } (x_n,y_n)\rightarrow(x_{n+1},y_n)\rightarrow(x_{n+1},y_{n+1}) \mbox{ as follows: }\ \left\{\begin{array}{l} x_{n+1}\in\mbox{argmin} \{f(\xi)+\beta_n c(\xi,y_n)+\alpha_n h(x_n,\xi): \xi\in\mathcal X\},\ y_{n+1}\in\mbox{argmin} \{g(\eta)+\mu_n c(x_{n+1},\eta)+\nu_n k(y_n,\eta): \eta\in\mathcal Y\}. \end{array}\right.$ Their most original feature is the introduction of the terms $h:\mathcal X\times\mathcal X\mapsto\R^+$ and $k:\mathcal Y\times\mathcal Y\mapsto\R^+$ which are costs to change or to move (distance-like functions, relative entropies) accounting for various inertial, friction, or anchoring effects. These algorithms are studied in a general abstract framework. The introduction of the costs to change $h$ and $k$ leads to proximal minimizations with corresponding dissipative effects. As a result, the algorithms enjoy nice convergent properties. Coefficients $\alpha_n$, $\beta_n$, $\mu_n$, $\nu_n$ are nonnegative parameters. When taking $\alpha_n=\nu_n=0$ and quadratic costs on a Hilbert space, one recovers the classical alternating minimization algorithm, which itself is a natural extension of the alternating projection algorithm of von Neumann. A number of new significant results hold in general metric spaces. We pay particular attention to the following cases: (1.) $(\mathcal X,d_{\mathcal X})$ and $(\mathcal Y,d_{\mathcal Y})$ are complete metric spaces and $h\geq d_{\mathcal X}$, $k\geq d_{\mathcal Y}$ (“high local costs to move”); the algorithms then provide sequences that converge to Nash equilibria. (2.) $\mathcal X=\mathcal Y=\mathcal H$ is a Hilbert space, the costs to change are quadratic (“low local costs to move”) and the functions $f,g:\mathcal H\mapsto\R\cup\{+\infty\}$ are closed, convex, proper; then some of the classical convergence theorems for alternating convex minimization algorithms, including those of Acker and Prestel, are properly extended with original proofs.