Factor Analysis Using Delta-Rule Wake-Sleep Learning

Authors:
Radford M. Neal;Peter Dayan
Affiliations:
Department of Statistics and Department of Computer Science, University of Toronto, Toronto M5S 1A1, Canada;Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, 02139 U.S.A.
Venue:
Neural Computation
Year:
1997

Citing 6
Cited 0

Self-Organization in a Perceptual Network

Computer
Original Contribution: Efficient information transfer and anti-Hebbian neural networks

Neural Networks
The role of constraints in Hebbian learning

Neural Computation
The Helmholtz machine

Neural Computation
Varieties of Helmholtz machine

Neural Networks - 1996 Special issue: four major hypotheses in neuroscience
Dynamic Model of Visual Memory Pedicts Neural Response Properties in the Visual Cortex

Dynamic Model of Visual Memory Pedicts Neural Response Properties in the Visual Cortex

Quantified Score

Hi-index	0.00

Visualization

Abstract

We describe a linear network that models correlations between real-valued visible variables using one or more real-valued hidden variables---a factor analysis model. This model can be seen as a linear version of the Helmholtz machine, and its parameters can be learned using the wake sleep method, in which learning of the primary generative model is as sisted by a recognition model, whose role is to fill in the values of hidden variables based on the values of visible variables. The generative and recognition models are jointly learned in wake and sleep phases, using just the delta rule. This learning procedure is comparable in simplicity to Hebbian learning, which produces a somewhat different representation of correlations in terms of principal components. We argue that the simplicity of wake-sleep learning makes factor analysis a plausible alternative to Hebbian learning as a model of activity-dependent cortical plasticity.