Online local learning algorithms for linear discriminant analysis
Pattern Recognition Letters - Special issue: Advances in pattern recognition
Theoretical Computer Science
Global Convergence of a PCA Learning Algorithm with a Constant Learning Rate
Computers & Mathematics with Applications
A unified learning algorithm to extract principal and minor components
Digital Signal Processing
Robotics and Computer-Integrated Manufacturing
| Hi-index | 35.68 |
The authors describe a neural network model (APEX) for multiple principal component extraction. All the synaptic weights of the model are trained with the normalized Hebbian learning rule. The network structure features a hierarchical set of lateral connections among the output units which serve the purpose of weight orthogonalization. This structure also allows the size of the model to grow or shrink without need for retraining the old units. The exponential convergence of the network is formally proved while there is significant performance improvement over previous methods. By establishing an important connection with the recursive least squares algorithm they have been able to provide the optimal size for the learning step-size parameter which leads to a significant improvement in the convergence speed. This is in contrast with previous neural PCA models which lack such numerical advantages. The APEX algorithm is also parallelizable allowing the concurrent extraction of multiple principal components. Furthermore, APEX is shown to be applicable to the constrained PCA problem where the signal variance is maximized under external orthogonality constraints. They then study various principal component analysis (PCA) applications that might benefit from the adaptive solution offered by APEX. In particular they discuss applications in spectral estimation, signal detection and image compression and filtering, while other application domains are also briefly outlined