Minimization methods for non-differentiable functions
Minimization methods for non-differentiable functions
Parallel and distributed computation: numerical methods
Parallel and distributed computation: numerical methods
A bridging model for parallel computation
Communications of the ACM
On the limited memory BFGS method for large scale optimization
Mathematical Programming: Series A and B
SIAM Journal on Control and Optimization
Scenarios and policy aggregation in optimization under uncertainty
Mathematics of Operations Research
Proximal minimization algorithm with D-functions
Journal of Optimization Theory and Applications
Nonlinear total variation based noise removal algorithms
Proceedings of the eleventh annual international conference of the Center for Nonlinear Studies on Experimental mathematics : computational issues in nonlinear science: computational issues in nonlinear science
Mathematical Programming: Series A and B
Nonlinear proximal point algorithms using Bregman functions, with applications to convex programming
Mathematics of Operations Research
Parallel alternating direction multiplier decomposition of convex programs
Journal of Optimization Theory and Applications
A proximal-based decomposition method for convex minimization problems
Mathematical Programming: Series A and B
Dykstra's alternating projection algorithm for two sets
Journal of Approximation Theory
A limited memory algorithm for bound constrained optimization
SIAM Journal on Scientific Computing
The nature of statistical learning theory
The nature of statistical learning theory
On convergence of an augmented Lagrangian decomposition method for sparse convex optimization
Mathematics of Operations Research
Alternating directions methods for the parallel solution of large-scale block-structured optimization problems
Applied numerical linear algebra
Applied numerical linear algebra
A variable-penalty alternating directions method for convex optimization
Mathematical Programming: Series A and B
A Modified Forward-Backward Splitting Method for Maximal Monotone Mappings
SIAM Journal on Control and Optimization
Basic Linear Algebra Subprograms for Fortran Usage
ACM Transactions on Mathematical Software (TOMS)
Journal of Optimization Theory and Applications
Parallel Optimization: Theory, Algorithms and Applications
Parallel Optimization: Theory, Algorithms and Applications
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Atomic Decomposition by Basis Pursuit
SIAM Review
Alternating Projection-Proximal Methods for Convex Programming and Variational Inequalities
SIAM Journal on Optimization
A decision-theoretic generalization of on-line learning and an application to boosting
EuroCOLT '95 Proceedings of the Second European Conference on Computational Learning Theory
SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
Convex Optimization
A family of projective splitting methods for the sum of two maximal monotone operators
Mathematical Programming: Series A and B
An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression
The Journal of Machine Learning Research
MapReduce: simplified data processing on large clusters
Communications of the ACM - 50th anniversary issue: 1958 - 2008
Bigtable: A Distributed Storage System for Structured Data
ACM Transactions on Computer Systems (TOCS)
The Journal of Machine Learning Research
Algorithm 887: CHOLMOD, Supernodal Sparse Cholesky Factorization and Update/Downdate
ACM Transactions on Mathematical Software (TOMS)
Graphical Models, Exponential Families, and Variational Inference
Foundations and Trends® in Machine Learning
The Unreasonable Effectiveness of Data
IEEE Intelligent Systems
Distributed in-network channel decoding
IEEE Transactions on Signal Processing
Smooth Optimization Approach for Sparse Covariance Selection
SIAM Journal on Optimization
General Projective Splitting Methods for Sums of Maximal Monotone Operators
SIAM Journal on Control and Optimization
SIAM Review
Bregman Iterative Algorithms for $\ell_1$-Minimization with Applications to Compressed Sensing
SIAM Journal on Imaging Sciences
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
SIAM Journal on Imaging Sciences
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
SIAM Journal on Imaging Sciences
The Split Bregman Method for L1-Regularized Problems
SIAM Journal on Imaging Sciences
Removing Multiplicative Noise by Douglas-Rachford Splitting Methods
Journal of Mathematical Imaging and Vision
Bundle Methods for Regularized Risk Minimization
The Journal of Machine Learning Research
Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning
Brief paper: Segmentation of ARX-models using sum-of-norms regularization
Automatica (Journal of IFAC)
Pregel: a system for large-scale graph processing
Proceedings of the 2010 ACM SIGMOD International Conference on Management of data
Monotone operator splitting for optimization problems in sparse recovery
ICIP'09 Proceedings of the 16th IEEE international conference on Image processing
Distributed consensus-based demodulation: algorithms and error analysis
IEEE Transactions on Wireless Communications
Design patterns for efficient graph algorithms in MapReduce
Proceedings of the Eighth Workshop on Mining and Learning with Graphs
Consensus-Based Distributed Support Vector Machines
The Journal of Machine Learning Research
Spark: cluster computing with working sets
HotCloud'10 Proceedings of the 2nd USENIX conference on Hot topics in cloud computing
Fast image recovery using variable splitting and constrained optimization
IEEE Transactions on Image Processing
Distributed sparse linear regression
IEEE Transactions on Signal Processing
HaLoop: efficient iterative data processing on large clusters
Proceedings of the VLDB Endowment
Restoration of Poissonian images using alternating direction optimization
IEEE Transactions on Image Processing
Hadoop: The Definitive Guide
SIAM Journal on Scientific Computing
Fully Distributed Algorithms for Convex Optimization Problems
SIAM Journal on Optimization
Consensus in Ad Hoc WSNs With Noisy Links—Part I: Distributed Estimation of Deterministic Signals
IEEE Transactions on Signal Processing
IEEE Transactions on Signal Processing
IEEE Transactions on Information Theory
IEEE Transactions on Information Theory
Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?
IEEE Transactions on Information Theory
De-noising by soft-thresholding
IEEE Transactions on Information Theory
IEEE Transactions on Image Processing
Emerging topic detection using dictionary learning
Proceedings of the 20th ACM international conference on Information and knowledge management
Convex and Network Flow Optimization for Structured Sparsity
The Journal of Machine Learning Research
Commute time guided transformation for feature extraction
Computer Vision and Image Understanding
Optimization with Sparsity-Inducing Penalties
Foundations and Trends® in Machine Learning
On the $O(1/n)$ Convergence Rate of the Douglas-Rachford Alternating Direction Method
SIAM Journal on Numerical Analysis
Structured sparsity via alternating direction methods
The Journal of Machine Learning Research
Registration for correlative microscopy using image analogies
WBIR'12 Proceedings of the 5th international conference on Biomedical Image Registration
Distributed customer behavior prediction using multiplex data: A collaborative MK-SVM approach
Knowledge-Based Systems
Journal of Computational Neuroscience
Finding correspondence from multiple images via sparse and low-rank decomposition
ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part V
Sparse methods for biomedical data
ACM SIGKDD Explorations Newsletter
Non-convex optimization on stiefel manifold and applications to machine learning
ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part I
Group sparse inverse covariance selection with a dual augmented lagrangian method
ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part III
An ADM-based splitting method for separable convex programming
Computational Optimization and Applications
Learning with infinitely many features
Machine Learning
Temperature aware workload management in geo-distributed datacenters
Proceedings of the ACM SIGMETRICS/international conference on Measurement and modeling of computer systems
Computational Optimization and Applications
Human reidentification with transferred metric learning
ACCV'12 Proceedings of the 11th Asian conference on Computer Vision - Volume Part I
An efficient ADMM algorithm for multidimensional anisotropic total variation regularization problems
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Robust principal component analysis via capped norms
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Querying discriminative and representative samples for batch mode active learning
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Distributed large-scale natural graph factorization
Proceedings of the 22nd international conference on World Wide Web
Multisample aCGH Data Analysis via Total Variation and Spectral Regularization
IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
Estimating building simulation parameters via Bayesian structure learning
Proceedings of the 2nd International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications
Proceedings of the 2nd International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications
Learning a factor model via regularized PCA
Machine Learning
Fine-grained semi-supervised labeling of large shape collections
ACM Transactions on Graphics (TOG)
Sparse localized deformation components
ACM Transactions on Graphics (TOG)
A novel sparse group Gaussian graphical model for functional connectivity estimation
IPMI'13 Proceedings of the 23rd international conference on Information Processing in Medical Imaging
Total variation regularization algorithms for images corrupted with different noise models: a review
Journal of Electrical and Computer Engineering
Efficient kernel learning from side information using ADMM
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Accurate probability calibration for multiple classifiers
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Audio classification with low-rank matrix representation features
ACM Transactions on Intelligent Systems and Technology (TIST) - Special Section on Intelligent Mobile Knowledge Discovery and Management Systems and Special Issue on Social Web Mining
Two-stage stochastic optimization for optimal power flow under renewable generation uncertainty
ACM Transactions on Modeling and Computer Simulation (TOMACS) - Special issue on simulation in complex service systems
LASER: a scalable response prediction platform for online advertising
Proceedings of the 7th ACM international conference on Web search and data mining
Cluster analysis: unsupervised learning via supervised learning with a non-convex penalty
The Journal of Machine Learning Research
The Journal of Machine Learning Research
Stationary-sparse causality network learning
The Journal of Machine Learning Research
Variational algorithms for marginal MAP
The Journal of Machine Learning Research
Personalized collaborative clustering
Proceedings of the 23rd international conference on World wide web
Robust subspace discovery via relaxed rank minimization
Neural Computation
Poisson Noise Reduction with Non-local PCA
Journal of Mathematical Imaging and Vision
A Combined First and Second Order Variational Approach for Image Reconstruction
Journal of Mathematical Imaging and Vision
Homogeneous Penalizers and Constraints in Convex Image Restoration
Journal of Mathematical Imaging and Vision
Path-following gradient-based decomposition algorithms for separable convex optimization
Journal of Global Optimization
Sparse iterative closest point
SGP '13 Proceedings of the Eleventh Eurographics/ACMSIGGRAPH Symposium on Geometry Processing
Consistent shape maps via semidefinite programming
SGP '13 Proceedings of the Eleventh Eurographics/ACMSIGGRAPH Symposium on Geometry Processing
Novel document detection for massive data streams using distributed dictionary learning
IBM Journal of Research and Development
Hi-index | 0.00 |
Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for ℓ1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.