perceptrons, backpropagation, batch training, training sets
Multilayer perceptrons are often trained using error backpropagation (BP). BP training can be done in either a batch or continuous manner. Claims have frequently been made that batch training is faster and/or more "correct" than continuous training because it uses a better approximation of the true gradient for its weight updates. These claims are often supported by empirical evidence on very small data sets. These claims are untrue, however, for large training sets. This paper explains why batch training is much slower than continuous training for large training sets. Various levels of semi-batch training used on a 20,000-instance speech recognition task show a roughly linear increase in training time required with an increase in batch size.
Original Publication Citation
Wilson, D. R. and Martinez, T. R., "The Inefficiency of Batch Training on Large Training Sets", Proceedings of the IEEE International Joint Conference on Neural Networks IJCNN', Vol. 2, pp. 113-117, 2.
BYU ScholarsArchive Citation
Martinez, Tony R. and Wilson, D. Randall, "The Inefficiency of Batch Training for Large Training Sets" (2000). All Faculty Publications. 1109.
Physical and Mathematical Sciences
© 2000 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Copyright Use Information