I.E. Livieris, D.G. Sotiropoulos and P. Pintelas. On descent spectral CG algorithms for training recurrent neural networks. In: Proceedings of 13th Panhellinic Conference on Informatics (PCI'09), Corfu, Greece, pages 65-69, 2009.


Abstract -  In this paper, we evaluate the performance of descent conjugate gradient methods and we propose a new algorithm for training recurrent neural networks. The presented algorithm preserves the advantages of classical conjugate gradient methods while simultaneously avoids the usually inefficient restarts. Simulation results are also presented using three different recurrent neural network architectures in a variety of benchmarks.