Convergence of alternating optimization

Authors:
James C. Bezdek;Richard J. Hathaway
Affiliations:
Computer Science Department, University of West Florida, Pensacola, FL;Department of Mathematical Sciences, Georgia Southern University, Statesboro, GA
Venue:
Neural, Parallel & Scientific Computations
Year:
2003

Citing 12
Cited 23

Local convergence analysis of a grouped variable version of coordinate descent

Journal of Optimization Theory and Applications
Convergence theory for fuzzy c-means: counterexamples and repairs

IEEE Transactions on Systems, Man and Cybernetics
Grouped coordinate minimization using Newton's method for inexact minimization in one vector coordinate

Journal of Optimization Theory and Applications
Iterative solution of nonlinear equations in several variables

Iterative solution of nonlinear equations in several variables
Local convergence of tri-level alternating optimization

Neural, Parallel & Scientific Computations
Fuzzy Models and Algorithms for Pattern Recognition and Image Processing

Fuzzy Models and Algorithms for Pattern Recognition and Image Processing
Convergence of a block coordinate descent method for nondifferentiable minimization

Journal of Optimization Theory and Applications
On efficiency of optimization in fuzzy c-means

Neural, Parallel & Scientific Computations
Some Notes on Alternating Optimization

AFSS '02 Proceedings of the 2002 AFSS International Conference on Fuzzy Systems. Calcutta: Advances in Soft Computing
A contribution to convergence theory of fuzzy c-means and derivatives

IEEE Transactions on Fuzzy Systems
Quantization

IEEE Transactions on Information Theory
On the convergence of the block nonlinear Gauss-Seidel method under convex constraints

Operations Research Letters

A fast iterative algorithm for fisher discriminant using heterogeneous kernels

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Active learning via transductive experimental design

ICML '06 Proceedings of the 23rd international conference on Machine learning
Approximate data mining in very large relational data

ADC '06 Proceedings of the 17th Australasian Database Conference - Volume 49
Maximum margin clustering made practical

Proceedings of the 24th international conference on Machine learning
Automatic image annotation via local multi-label classification

CIVR '08 Proceedings of the 2008 international conference on Content-based image and video retrieval
Polyhedral classifier for target detection: a case study: colorectal cancer

Proceedings of the 25th international conference on Machine learning
An Improved Multi-task Learning Approach with Applications in Medical Diagnosis

ECML PKDD '08 Proceedings of the 2008 European Conference on Machine Learning and Knowledge Discovery in Databases - Part I
A mathematical programming formulation for sparse collaborative computer aided diagnosis

AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 1
Transferring multi-device localization models using latent multi-task learning

AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 3
Distance metric learning from uncertain side information with application to automated photo tagging

MM '09 Proceedings of the 17th ACM international conference on Multimedia
Extending fuzzy and probabilistic clustering to very large data sets

Computational Statistics & Data Analysis
Maximum margin clustering made practical

IEEE Transactions on Neural Networks
Multi-class classifiers and their underlying shared structure

IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Distance metric learning from uncertain side information for automated photo tagging

ACM Transactions on Intelligent Systems and Technology (TIST)
A Multilevel Algorithm for Simultaneously Denoising and Deblurring Images

SIAM Journal on Scientific Computing
Building sparse support vector machines for multi-instance classification

ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part I
Maximin initialization for cluster analysis

CIARP'06 Proceedings of the 11th Iberoamerican conference on Progress in Pattern Recognition, Image Analysis and Applications
Curve fitting for short time series data from high throughput experiments with correction for biological variation

IDA'12 Proceedings of the 11th international conference on Advances in Intelligent Data Analysis
X-Ray CT Image Reconstruction via Wavelet Frame Based Regularization and Radon Domain Inpainting

Journal of Scientific Computing
Querying discriminative and representative samples for batch mode active learning

Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Locally discriminative spectral clustering with composite manifold

Neurocomputing
Robust unsupervised feature selection

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Multiview Hessian discriminative sparse coding for image annotation

Computer Vision and Image Understanding

Quantified Score

Hi-index	0.00

Visualization

Abstract

Let f : Rs → R be a real-valued function, and let x = (x1,...,xs)T ∈ Rs be partitioned into t subsets of non-overlapping variables as x = (X1,...,Xt)T, with Xi ∈ Rpi for i = 1,...,t, Σi=1tpi = s. Alternating optimization (AO) is an iterative procedure for minimizing f(x) = f(X1, X2,..., Xt) jointly over all variables by alternating restricted minimizations over the individual subsets of variables X1,...., Xt. Alternating optimization has been (more or less) studied and used in a wide variety of areas. Here a self-contained and general convergence theory is presented that is applicable to all partitionings of x. Under reasonable assumptions, the general AO approach is shown to be locally, q-linearly convergent, and to also exhibit a type of global convergence.