Maximum margin clustering made practical

Authors:
Kai Zhang;Ivor W. Tsang;James T. Kwok
Affiliations:
Life Science Division, Lawrence Berkeley National Laboratory, Berkeley, CA and Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong;School of Computer Engineering, Nanyang Technological University, Singapore, Singapore and Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong;Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong
Venue:
IEEE Transactions on Neural Networks
Year:
2009

Citing 27
Cited 10

Algorithms for clustering data

Algorithms for clustering data
Vector quantization and signal compression

Vector quantization and signal compression
The nature of statistical learning theory

The nature of statistical learning theory
Fast training of support vector machines using sequential minimal optimization

Advances in kernel methods
Accelerating exact k-means algorithms with geometric reasoning

KDD '99 Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining
Least Squares Support Vector Machine Classifiers

Neural Processing Letters
Normalized Cuts and Image Segmentation

IEEE Transactions on Pattern Analysis and Machine Intelligence
A D. C. Optimization Algorithm for Solving the Trust-Region Subproblem

SIAM Journal on Optimization
An Efficient k-Means Clustering Algorithm: Analysis and Implementation

IEEE Transactions on Pattern Analysis and Machine Intelligence
The concave-convex procedure

Neural Computation
Efficient svm training using low-rank kernel representations

The Journal of Machine Learning Research
Spectral Grouping Using the Nyström Method

IEEE Transactions on Pattern Analysis and Machine Intelligence
Convergence of alternating optimization

Neural, Parallel & Scientific Computations
Unsupervised word sense disambiguation rivaling supervised methods

ACL '95 Proceedings of the 33rd annual meeting on Association for Computational Linguistics
Convex Optimization

Convex Optimization
Clustering and Information Retrieval (Network Theory and Applications)

Clustering and Information Retrieval (Network Theory and Applications)
Learning the Kernel Matrix with Semidefinite Programming

The Journal of Machine Learning Research
A new approach to data driven clustering

ICML '06 Proceedings of the 23rd international conference on Machine learning
Trading convexity for scalability

ICML '06 Proceedings of the 23rd international conference on Machine learning
Clustering documents with an exponential-family approximation of the Dirichlet compound multinomial distribution

ICML '06 Proceedings of the 23rd international conference on Machine learning
Clustering graphs by weighted substructure mining

ICML '06 Proceedings of the 23rd international conference on Machine learning
Discriminative unsupervised learning of structured predictors

ICML '06 Proceedings of the 23rd international conference on Machine learning
Large Scale Transductive SVMs

The Journal of Machine Learning Research
Maximum margin clustering made practical

Proceedings of the 24th international conference on Machine learning
Large-Scale Kernel Machines (Neural Information Processing)

Large-Scale Kernel Machines (Neural Information Processing)
Unsupervised and semi-supervised multi-class support vector machines

AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
An improved conjugate gradient scheme to the solution of least squares SVM

IEEE Transactions on Neural Networks

Novel maximum-margin training algorithms for supervised neural networks

IEEE Transactions on Neural Networks
Serendipitous learning: learning beyond the predefined label space

Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining
Learning from positive and unlabelled examples using maximum margin clustering

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part III
Soft large margin clustering

Information Sciences: an International Journal
Unsupervised non-parametric kernel learning algorithm

Knowledge-Based Systems
Attribute-augmented semantic hierarchy: towards bridging semantic gap and intention gap in image retrieval

Proceedings of the 21st ACM international conference on Multimedia
Maximum margin clustering for state decomposition of metastable systems

IWANN'13 Proceedings of the 12th international conference on Artificial Neural Networks: advances in computational intelligence - Volume Part I
A bagging SVM to learn from positive and unlabeled examples

Pattern Recognition Letters
Convex and scalable weakly labeled SVMs

The Journal of Machine Learning Research
Variational learning of finite Dirichlet mixture models using component splitting

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Motivated by the success of large margin methods in supervised learning, maximum margin clustering (MMC) is a recent approach that aims at extending large margin methods to unsupervised learning. However, its optimization problem is nonconvex and existing MMC methods all rely on reformulating and relaxing the nonconvex optimization problem as semidefinite programs (SDP). Though SDP is convex and standard solvers are available, they are computationally very expensive and only small data sets can be handled. To make MMC more practical, we avoid SDP relaxations and propose in this paper an efficient approach that performs alternating optimization directly on the original nonconvex problem. A key step to avoid premature convergence in the resultant iterative procedure is to change the loss function from the hinge loss to the Laplacian/square loss so that overconfident predictions are penalized. Experiments on a number of synthetic and real-world data sets demonstrate that the proposed approach is more accurate, much faster (hundreds to tens of thousands of times faster), and can handle data sets that are hundreds of times larger than the largest data set reported in the MMC literature.