Unsupervised learning with stochastic gradient

Authors:
Harold Szu;Ivica Kopriva
Affiliations:
Department of Electrical and Computer Engineering, George Washington University, 801, 22nd St. NW, Washington, DC 20052, USA;Department of Electrical and Computer Engineering, George Washington University, 801, 22nd St. NW, Washington, DC 20052, USA
Venue:
Neurocomputing
Year:
2005

Citing 13
Cited 1

Blind separation of sources, Part 1: an adaptive algorithm based on neuromimetic architecture

Signal Processing
Original Contribution: Blind separation of sources: A nonlinear neural algorithm

Neural Networks
Independent component analysis, a new concept?

Signal Processing - Special issue on higher order statistics
An information-maximization approach to blind separation and blind deconvolution

Neural Computation
Neuro-fuzzy and soft computing: a computational approach to learning and machine intelligence

Neuro-fuzzy and soft computing: a computational approach to learning and machine intelligence
A fast fixed-point algorithm for independent component analysis

Neural Computation
Natural gradient works efficiently in learning

Neural Computation
Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications

Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
Source separation in post-nonlinear mixtures

IEEE Transactions on Signal Processing
Blind separation of mixture of independent sources through aquasi-maximum likelihood approach

IEEE Transactions on Signal Processing
Equivariant adaptive source separation

IEEE Transactions on Signal Processing
Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images

IEEE Transactions on Pattern Analysis and Machine Intelligence

Neural networks letter: Capturing significant events with neural networks

Neural Networks

Quantified Score

Hi-index	0.01

Visualization

Abstract

A stochastic gradient is formulated based on deterministic gradient augmented with Cauchy simulated annealing capable to reach a global minimum with a convergence speed significantly faster when simulated annealing is used alone. In order to solve space-time variant inverse problems known as blind source separation, a novel Helmholtz free energy contrast function, H=E-T"0S, with imposed thermodynamics constraint at a constant temperature T"0 was introduced generalizing the Shannon maximum entropy S of the closed systems to the open systems having non-zero input-output energy exchange E. Here, only the input data vector was known while source vector and mixing matrix were unknown. A stochastic gradient was successfully applied to solve inverse space-variant imaging problems on a concurrent pixel-by-pixel basis with the unknown mixing matrix (imaging point spread function) varying from pixel to pixel.