Multi-stage multi-task feature learning

Authors:
Pinghua Gong;Jieping Ye;Changshui Zhang
Affiliations:
State Key Laboratory on Intelligent Technology and Systems, Tsinghua National Laboratory for Information Science and Technology, Department of Automation, Tsinghua University, Beijing, China;Computer Science and Engineering, Center for Evolutionary Medicine and Informatics, The Biodesign Institute, Arizona State University, Tempe, AZ;State Key Laboratory on Intelligent Technology and Systems, Tsinghua National Laboratory for Information Science and Technology, Department of Automation, Tsinghua University, Beijing, China
Venue:
The Journal of Machine Learning Research
Year:
2013

Citing 18
Cited 0

Multitask Learning

Machine Learning - Special issue on inductive transfer
Convergence of a block coordinate descent method for nondifferentiable minimization

Journal of Optimization Theory and Applications
The concave-convex procedure

Neural Computation
Regularized multi--task learning

Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining
Learning Gaussian processes from multiple tasks

ICML '05 Proceedings of the 22nd international conference on Machine learning
An Improved Multi-task Learning Approach with Applications in Medical Diagnosis

ECML PKDD '08 Proceedings of the 2008 European Conference on Machine Learning and Knowledge Discovery in Databases - Part I
Convex multi-task feature learning

Machine Learning
Sparse reconstruction by separable approximation

IEEE Transactions on Signal Processing
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems

SIAM Journal on Imaging Sciences
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems

SIAM Journal on Imaging Sciences
Analysis of Multi-stage Convex Relaxation for Sparse Regularization

The Journal of Machine Learning Research
Multi-task feature learning via efficient l2, 1-norm minimization

UAI '09 Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence
Learning incoherent sparse and low-rank patterns from multiple tasks

Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining
Union Support Recovery in Multi-task Learning

The Journal of Machine Learning Research
Decoding by linear programming

IEEE Transactions on Information Theory
Stable recovery of sparse overcomplete representations in the presence of noise

IEEE Transactions on Information Theory
On the convergence of the block nonlinear Gauss-Seidel method under convex constraints

Operations Research Letters
Robust multi-task feature learning

Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining

Quantified Score

Hi-index	0.00

Visualization

Abstract

Multi-task sparse feature learning aims to improve the generalization performance by exploiting the shared features among tasks. It has been successfully applied to many applications including computer vision and biomedical informatics. Most of the existing multi-task sparse feature learning algorithms are formulated as a convex sparse regularization problem, which is usually suboptimal, due to its looseness for approximating an l0-type regularizer. In this paper, we propose a non-convex formulation for multi-task sparse feature learning based on a novel nonconvex regularizer. To solve the non-convex optimization problem, we propose a Multi-Stage Multi-Task Feature Learning (MSMTFL) algorithm; we also provide intuitive interpretations, detailed convergence and reproducibility analysis for the proposed algorithm. Moreover, we present a detailed theoretical analysis showing that MSMTFL achieves a better parameter estimation error bound than the convex formulation. Empirical studies on both synthetic and real-world data sets demonstrate the effectiveness of MSMTFL in comparison with the state of the art multi-task sparse feature learning algorithms.