Introduction to statistical pattern recognition (2nd ed.)
Introduction to statistical pattern recognition (2nd ed.)
Fundamentals of matrix computations
Fundamentals of matrix computations
LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares
ACM Transactions on Mathematical Software (TOMS)
Algorithm 583: LSQR: Sparse Linear Equations and Least Squares Problems
ACM Transactions on Mathematical Software (TOMS)
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Kernel independent component analysis
The Journal of Machine Learning Research
RCV1: A New Benchmark Collection for Text Categorization Research
The Journal of Machine Learning Research
A Two-Stage Linear Discriminant Analysis via QR-Decomposition
IEEE Transactions on Pattern Analysis and Machine Intelligence
The Journal of Machine Learning Research
Higher order learning with graphs
ICML '06 Proceedings of the 23rd international conference on Machine learning
Pattern Recognition and Machine Learning (Information Science and Statistics)
Pattern Recognition and Machine Learning (Information Science and Statistics)
SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis
IEEE Transactions on Knowledge and Data Engineering
A least squares formulation for canonical correlation analysis
Proceedings of the 25th international conference on Machine learning
Hypergraph spectral learning for multi-label classification
Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining
A least squares formulation for a class of generalized eigenvalue problems in machine learning
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
On the equivalence between canonical correlation analysis and orthonormalized partial least squares
IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Spectral regression: a regression framework for efficient regularized subspace learning
Spectral regression: a regression framework for efficient regularized subspace learning
Overview and recent advances in partial least squares
SLSFS'05 Proceedings of the 2005 international conference on Subspace, Latent Structure and Feature Selection
Least squares online linear discriminant analysis
Expert Systems with Applications: An International Journal
A Rayleigh-Ritz style method for large-scale discriminant analysis
Pattern Recognition
Soft label based Linear Discriminant Analysis for image recognition and retrieval
Computer Vision and Image Understanding
Hi-index | 0.00 |
Dimensionality reduction plays an important role in many data mining applications involving high-dimensional data. Many existing dimensionality reduction techniques can be formulated as a generalized eigenvalue problem, which does not scale to large-size problems. Prior work transforms the generalized eigenvalue problem into an equivalent least squares formulation, which can then be solved efficiently. However, the equivalence relationship only holds under certain assumptions without regularization, which severely limits their applicability in practice. In this paper, an efficient two-stage approach is proposed to solve a class of dimensionality reduction techniques, including Canonical Correlation Analysis, Orthonormal Partial Least Squares, linear Discriminant Analysis, and Hypergraph Spectral Learning. The proposed two-stage approach scales linearly in terms of both the sample size and data dimensionality. The main contributions of this paper include (1) we rigorously establish the equivalence relationship between the proposed two-stage approach and the original formulation without any assumption; and (2) we show that the equivalence relationship still holds in the regularization setting. We have conducted extensive experiments using both synthetic and real-world data sets. Our experimental results confirm the equivalence relationship established in this paper. Results also demonstrate the scalability of the proposed two-stage approach.