• Home
  • Search
  • Towards a Very Fast Feedforward Multilayer Neural Networks Training Algorithm
  • Cite Icon17
  • https://doi.org/10.2478/jaiscr-2022-0012Copy DOI Icon

Towards a Very Fast Feedforward Multilayer Neural Networks Training Algorithm

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Abstract ** This paper presents a novel fast algorithm for feedforward neural networks training. It is based on the Recursive Least Squares (RLS) method commonly used for designing adaptive filters. Besides, it utilizes two techniques of linear algebra, namely the orthogonal transformation method, called the Givens Rotations (GR), and the QR decomposition, creating the GQR (symbolically we write GR + QR = GQR) procedure for solving the normal equations in the weight update process. In this paper, a novel approach to the GQR algorithm is presented. The main idea revolves around reducing the computational cost of a single rotation by eliminating the square root calculation and reducing the number of multiplications. The proposed modification is based on the scaled version of the Givens rotations, denoted as SGQR. This modification is expected to bring a significant training time reduction comparing to the classic GQR algorithm. The paper begins with the introduction and the classic Givens rotation description. Then, the scaled rotation and its usage in the QR decomposition is discussed. The main section of the article presents the neural network training algorithm which utilizes scaled Givens rotations and QR decomposition in the weight update process. Next, the experiment results of the proposed algorithm are presented and discussed. The experiment utilizes several benchmarks combined with neural networks of various topologies. It is shown that the proposed algorithm outperforms several other commonly used methods, including well known Adam optimizer.

Similar Papers
  • Preprint Article

Neural Network Based Recursive Least Square Technique for Indoor Wireless Positioning

  • Aug 29, 2023
  • Bhagawat Adhikari
  • Research Article
  • Citations3

Efficient Recursive Least Squares Methods forthe CMAC Neural Network

  • Jan 01, 2011
  • International Journal of Machine Learning and Computing
  • C Laufer +1
  • Conference Article
  • Citations3

A Modified Recursive Least Squares Algorithm with Forgetting and Bounded Covariance

  • Jul 01, 2019
  • Adam L Bruce +1
  • Preprint Article

A Neural Network Based Recursive Least Square Multilateration Technique for Indoor Positioning

  • Nov 21, 2024
  • Bhagawat Adhikari +1
  • Book Chapter
  • Citations2

A New Variant of the GQR Algorithm for Feedforward Neural Networks Training

  • Jan 01, 2021
  • Jarosław Bilski +1
  • Conference Article
  • Citations8

Online Model Identification Method of Vanadium Redox Flow Battery Based on Time-varying Forgetting Factor Recursive Least Squares

  • Nov 01, 2019
  • Miaoyun Sun +4
  • Research Article
  • Citations3

A nonlinear recursive least squares algorithm for crosstalk‐resistant noise canceler

  • Jan 15, 2003
  • Electronics and Communications in Japan (Part III: Fundamental Electronic Science)
  • Masaaki Umayahara +2
  • Research Article
  • Citations56

Envelope Tracking Techniques for Flicker Mitigation and Voltage Regulation

  • Oct 01, 2004
  • IEEE Transactions on Power Delivery
  • M.I Marei +2
  • Conference Article

A new algorithm based on RLS for voiced/unvoiced segmentation

  • Dec 01, 2008
  • Weibo Xie +2
  • Conference Article
  • Citations1

Comparison between Epsilon Normalized Least Means Square (ϵ-NLMS) and Recursive Least Squares (RLS) Adaptive Algorithms

  • Mar 01, 2018
  • Ali Salah Mahdi +2
  • Research Article
  • Citations7

Learning and Controlling Multiscale Dynamics in Spiking Neural Networks Using Recursive Least Square Modifications.

  • Aug 01, 2024
  • IEEE transactions on cybernetics
  • Qinglai Wei +2
  • Book Chapter
  • Citations13

Non-Linear Adaptive Prediction of Speech with a Pipelined Recurrent Neural Network and Advanced Learning Algorithms

  • Jan 01, 1998
  • Danilo Mandic +2
  • Research Article
  • Citations1

Stabilizing the training of deep neural networks using Adam optimization and gradient clipping

  • Jan 27, 2023
  • INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT
  • Rudra Tiwari
  • Conference Article
  • Citations34

Training of Artificial Neural Network Using PSO With Novel Initialization Technique

  • Nov 01, 2018
  • Hafiz Tayyab Rauf +3
  • Conference Article

Research on the state observer of the vehicle body slip angle based on the Recursive Least Squares

  • May 01, 2010
  • Liang Chu +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.