A fast parallel optimization for training support vector machine

Authors:
Jian-Xiong Dong;Adam Krzyżak;Ching Y. Suen
Affiliations:
Centre for Pattern Recognition and Machine Intelligence, Concordia University, Montreal, Quebec, Canada;Department of Computer Science, Concordia University, Montreal, Quebec, Canada;Centre for Pattern Recognition and Machine Intelligence, Concordia University, Montreal, Quebec, Canada
Venue:
MLDM'03 Proceedings of the 3rd international conference on Machine learning and data mining in pattern recognition
Year:
2003

Citing 9
Cited 12

A set of level 3 basic linear algebra subprograms

ACM Transactions on Mathematical Software (TOMS)
A nonlinear normalization method for handprinted Kanji character recognition—line density equalization

Pattern Recognition
Making large-scale support vector machine learning practical

Advances in kernel methods
Fast training of support vector machines using sequential minimal optimization

Advances in kernel methods
Training Invariant Support Vector Machines

Machine Learning
Text Categorization with Suport Vector Machines: Learning with Many Relevant Features

ECML '98 Proceedings of the 10th European Conference on Machine Learning
A Fast SVM Training Algorithm

SVM '02 Proceedings of the First International Workshop on Pattern Recognition with Support Vector Machines
Training Support Vector Machines: an Application to Face Detection

CVPR '97 Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97)
Improvements to Platt's SMO Algorithm for SVM Classifier Design

Neural Computation

An improved handwritten Chinese character recognition system using support vector machine

Pattern Recognition Letters - Special issue: Artificial neural networks in pattern recognition
Parallel Software for Training Large Scale Support Vector Machines on Multiprocessor Systems

The Journal of Machine Learning Research
On the optimal working set size in serial and parallel support vector machine learning with the decomposition algorithm

AusDM '06 Proceedings of the fifth Australasian conference on Data mining and analystics - Volume 61
Incorporating prior knowledge in support vector machines for classification: A review

Neurocomputing
Granular Kernel Trees with parallel Genetic Algorithms for drug activity comparisons

International Journal of Data Mining and Bioinformatics
Automatic clinical image segmentation using pathological modeling, PCA and SVM

Engineering Applications of Artificial Intelligence
Hybrid MPI/OpenMP Parallel Linear Support Vector Machine Training

The Journal of Machine Learning Research
HyParSVM: a new hybrid parallel software for support vector machine learning on SMP clusters

Euro-Par'06 Proceedings of the 12th international conference on Parallel Processing
A novel parallel reduced support vector machine

ICNC'05 Proceedings of the First international conference on Advances in Natural Computation - Volume Part I
Automatic clinical image segmentation using pathological modelling, PCA and SVM

MLDM'05 Proceedings of the 4th international conference on Machine Learning and Data Mining in Pattern Recognition
A MapReduce-based distributed SVM algorithm for automatic image annotation

Computers & Mathematics with Applications
Parallel multitask cross validation for Support Vector Machine using GPU

Journal of Parallel and Distributed Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

A fast SVM training algorithm for multi-classes consisting of parallel and sequential optimizations is presented. The main advantage of the parallel optimization step is to remove most non-support vectors quickly, which dramatically reduces the training time at the stage of sequential optimization. In addition, some strategies such as kernel caching, shrinking and calling BLAS functions are effectively integrated into the algorithm to speed up the training. Experiments on MNIST handwritten digit database have shown that, without sacrificing the generalization performance, the proposed algorithm has achieved a speed-up factor of 110, when compared with Keerthi et al.'s modified SMO. Moreover, for the first time ever we investigated the training performance of SVM on handwritten Chinese database ETL9B with more than 3000 categories and about 500,000 training samples. The total training time is just 5.1 hours. The raw error rate of 1.1% on ETL9B has been achieved.