Hopfield network, back-propagation, noisy associative training
A new approach to training a generalized Hopfield network is developed and evaluated in this work. Both the weight symmetricity constraint and the zero self-connection constraint are removed from standard Hopfield networks. Training is accomplished with Back-Propagation Through Time, using noisy versions of the memorized patterns. Training in this way is referred to as Noisy Associative Training (NAT). Performance of NAT is evaluated on both random and correlated data. NAT has been tested on several data sets, with a large number of training runs for each experiment. The data sets used include uniformly distributed random data and several data sets adapted from the U.C. Irvine Machine Learning Repository. Results show that for random patterns, Hopfield networks trained with NAT have an average overall recall accuracy 6.1 times greater than networks produced with either Hebbian or Pseudo-Inverse training. Additionally, these networks have 13% fewer spurious memories on average than networks trained with Pseudo-Inverse or Hebbian training. Typically, networks memorizing over 2N (where N is the number of nodes in the network) patterns are produced. Performance on correlated data shows an even greater improvement over networks produced with either Hebbian or Pseudo-Inverse training - An average of 27.8 times greater recall accuracy, with 14% fewer spurious memories.
Original Publication Citation
Clift, F. and Martinez, T. R., "Improved Hopfield Networks by Training with Noisy Data", Proceedings of the IEEE International Joint Conference on Neural Networks IJCNN'1, pp. 1138-1143, 21.
BYU ScholarsArchive Citation
Clift, Fred and Martinez, Tony R., "Improved Hopfield Networks by Training with Noisy Data" (2001). Faculty Publications. 1088.
Physical and Mathematical Sciences
© 2001 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Copyright Use Information