Self-consistent signal-to-noise analysis of Hopfield model with unit replacement

Authors:
Toru Aonishi;Yasunao Komatsu;Koji Kurata
Affiliations:
Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology, 4259-G5-17, Nagatuda-cho, Midori-ku, Yokohama, Kanagawa 226-8502, Japan;Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology, 4259-G5-17, Nagatuda-cho, Midori-ku, Yokohama, Kanagawa 226-8502, Japan;Faculty of Engineering, University of the Ryukyus, Nishibaru, Okinawa 903-0213, Japan
Venue:
Neural Networks
Year:
2010

Citing 3
Cited 1

Notions of associative memory and sparse coding

Neural Networks - 1996 Special issue: four major hypotheses in neuroscience
Rate models for conductance-based cortical neuronal networks

Neural Computation
Correlation Matrix Memories

IEEE Transactions on Computers

Memory capacity and information capacity of sparsely encoded associative memory with replacement units

Artificial Life and Robotics

Quantified Score

Hi-index	0.00

Visualization

Abstract

The Hopfield model has a storage capacity: the maximum number of memory patterns that can be stably stored. The memory state of this network model disappears if the number of embedded memory patterns is larger than 0.138N, where N is the system size. Recently, it has been shown in numerical simulations that the Hopfield model with a unit replacement process, in which a small number of old units are replaced with new ones at each learning step for embedding a new pattern, can stably retrieve recently embedded memory patterns even if an infinite number of patterns have been embedded. In this paper, we analyze the Hopfield model with the replacement process by utilizing self-consistent signal-to-noise analysis. We show that 3.21 is the minimum number of replaced units at each learning step that avoids an overload evoking disappearance of the memory state when embedding an infinite number of patterns. Furthermore, we show that the optimal number of replaced units at each learning step that maximizes the number of retrievable patterns is 6.95. These critical numbers of replaced units are independent of the system size N. Finally, we compare this model with the Hopfield model with the forgetting process.