A Reliable Resilient Backpropagation Method with Gradient Ascent

While the Resilient Backpropagation (RPROP) method can be extremely fast in converging to a solution, it suffers from the local minima problem. In this paper, a fast and reliable learning algorithm for multi-layer artificial neural networks is proposed. The learning model has two phases: the RPROP phase and the gradient ascent phase. The repetition of two phases can help the network get out of local minima. The proposed algorithm is tested on some benchmark problems. For all the above problems, the systems are shown to be capable of escaping from the local minima and converge faster than the Backpropagation with momentum algorithm and the simulated annealing techniques.

Convergence theorems for the kohonen feature mapping algorithms with VLRPs

This paper presents the development of several efficient learning algorithms for neural networks (ELEANNE). The ELEANNE 1 and ELEANNE 2 are two recursive least-squares learning algorithms, proposed for training single-layered neural networks with analog output. This paper also proposes a new optimization strategy for training single-layered neural networks, which provides the basis for the development of a variety of efficient learning algorithms. This optimization strategy is the source of the ELEANNE 3, a second-order learning algorithm for training single-layered neural networks with binary output. A simplified version of this algorithm, called ELEANNE 4, is also derived on the basis of some simplifying but reasonable assumptions. The two algorithms developed for single-layered neural networks provide the basis for the derivation of ELEANNE 5 and ELEANNE 6, which are proposed for training multilayered neural networks with binary output. The ELEANNE 7 is an efficient algorithm developed for training multilayered neural networks with either binary or analog output. >

This paper presents a novel Heuristic Global Learning (HER-GBL) algorithm for multilayer neural networks. The algorithm is based upon the least squares method to maintain the fast convergence speed, and the penalized optimization to solve the problem of local minima. The penalty term, defined as a Gaussian-type function of the weight, is to provide an uphill force to escape from local minima. As a result, the training performance is dramatically improved. The proposed HER-GBL algorithm yields excellent results in terms of convergence speed, avoidance of local minima and quality of solution.

Fractional calculus has been adopted in the modelling of many scientific processes and systems. Due to the inherent feature of long term memory of fractional derivatives, it has been used in the learning process of neural networks. A fractional order derivative based back propagation learning algorithm in neural networks is proposed in this paper. Specifically, Riemann-Liouville (R-L), Caputo (C) and Caputo Fabrizio (CF) fractional Derivative based on the back propagation algorithms in a three layer feed-forward neural network employed. To get a faster learning rate without oscillation, momentum factor is incorporated. The effect of fractional order and momentum factor is investigated and compared. The performance of these fractional derivatives based algorithms with integer derivatives based algorithm in terms of mean square error (MSE), particularly the salary based on years of experience is predicted. Results demonstrate that fractional derivative based learning algorithms outperform the integer derivatives.

According to Vision Indonesia, data on people with eye diseases in Indonesia in 2018-2019 were 3 million people or about 1.5% of the total population. So far, public information or knowledge about the recognition of eye disease disorders is still lacking. The problem in this study is how to educate the public about the introduction of eye diseases based on information on symptoms of the disease and how to apply the web-based Artificial Neural Network (ANN) algorithm for the introduction of eye diseases. The ANN algorithm in the eye disease recognition education system can conclude knowledge even though it does not have certainty and takes it into account sequentially so that the process is faster. In terms of educational content about eye disease recognition, this is a novelty to use. This research aims to create an educational system for introducing eye diseases based on information on symptoms of the disease and applying a web-based Artificial Neural Network (ANN) algorithm for the recognition of eye diseases. The method used is the Artificial Neural Network algorithm method. The work of ANN in the education system for the introduction of eye diseases is to make parameters of eye disease symptoms or indicators that will produce the type of eye disease. The research material used is data on types of eye diseases and symptoms of each type of eye disease. The research results are to create an education system that can help the public recognise eye diseases based on the symptoms of these eye diseases that can be run on a web platform. The Artificial Neural Network (ANN) algorithm can manage input analysis data from disease indicators and show the initial results of eye diseases that can be detected. suffered by someone based on Training Results Weights and Bias v11= 1.6769, v01= 0.4356, w11= -1.5233, w01= 0.3242. Based on white box testing, the test results are free from logical errors. The results of this study indicate that the use of the ANN algorithm for eye disease recognition shows accurate results based on eye disease symptom data.

Machine learning algorithms are used to analyze medical data sets effectively in present. Today machine learning gives us several necessary tools for intelligent data analysis and research. Especially in very recent years, the digital world has provided relatively inexpensive and available means to collect and store the data. The main aim is to implement supervised machine learning concept by using datasets regarding blood cells collected from blood cells detecting and counting sensors, of a human as the input which is trained by artificial neural network algorithm and apply decision tree classification learning algorithm to perform classification which results in recognizing and also possibly predict the disease based on the nature of blood cells and classify accordingly. Artificial Neural network algorithm seems to avoid pruning problem and has higher efficiency and accuracy in training the datasets. Also the using of Decision tree is because they are easy to interpret, understand and also possess non-linear characteristics between values. This holds well the performance of the tree constructed which gives better outputs. Beside all the application developed using machine learning in day today life, the use of such learning algorithm in such medical application will enhance and benefit the medical field.

Fast Learning Algorithms

Extreme learning machine (ELM) is an easy-to use and effective learning algorithm of single-hidden layer feed-forward neural networks (SLFNs). The classical learning algorithm in neural network, e. g. backpropagation, requires setting several user-defined parameters and may get into local minimum. However, ELM only requires setting the number of hidden neurons and the activation function. It does not require adjusting the input weights and hidden layer biases during the implementation of the algorithm, and it produces only one optimal solution. Therefore, ELM has the advantages of fast learning speed and good generalization performance. In this paper, ELM is introduced in predicting reservoir permeability. By comparing to SVM, we analyze its feasibility and advantages in reservoir permeability prediction. The experimental results show that ELM has similar accuracy compared to SVR, but it has obvious advantages in parameter selection and learning speed.

A highly efficient implementation of a backpropagation learning algorithm using matrix ISA

A fast and robust algorithm for training feedforward neural networks (FNNs) by using a variable forgetting factor and U-D factorization-based recursive prediction error (RPE) method is proposed. In comparison with the backpropagation (BP) and RPE based learning algorithms, the proposed algorithm, called UD-RPE, can provide much more accurate learning results in fewer iterations with fewer hidden nodes and improve convergence rate and numerical stability (robustness). In addition, it is less sensitive to start-up parameters, such as initial weights and initial covariance matrix, and the randomness in the observed data. It also has good generalization ability and needs less learning time. Simulation results of nonlinear dynamic system modeling and identification show that the algorithm proposed here is an effective and efficient learning algorithm for FNNs.

A new fast learning algorithm for a multilayer feedforward neural network is presented. The new algorithm, based on an innovative variable step size and gradient averaging, has excellent properties to overcome major drawbacks of the backpropagation algorithm and shows high convergence speed. The improved performance of the new algorithm in comparison with other algorithms is verified via computer simulations.

In this paper, a hybrid supervised learning algorithm for neural network was proposed. The problem of local minimum learning usually occurred in the real application of neural network is tried to be solved or reduced. In order to improve the efficiency and stability of conventional error back-propagation learning algorithm, a hybrid learning method combining the linear multi-regression and backpropagation techniques was developed. To demonstrate the superiority of the method we developed, one example was simulated. The conventional BP learning method was also performed as the comparison with the new method proposed. From the results shown, the conventional BP method easily makes neural model plunge into the local minima. On the contrary, the new method we proposed not only has a fast learning, but also has a better learning efficiency.

A new multi-layer artificial neural network learning algorithm based on pattern search method is proposed. The learning model has two phases-a pattern search phase, and a local minimum-escaping phase. In the pattern search phase, our method performs local search iteratively and minimize the error measure function along with the set of descent directions of the error measure directly and finds the nearest minima efficiently. When the network gets stuck in local minima, the local minimum-escaping phase attempts to fill up the valley by modifying temperature parameters in ascent direction of the error measure. Thus, the two phases are repeated until the network gets out of local minima. The learning model is designed to provide a very simple and effective means of searching the minima of objective function directly without any knowledge of its derivatives. We test this algorithm on benchmark problems, such as exclusive-or (XOR), parity, Arabic numerals recognition, function approximation problems and a real world classification task. For all problems, the systems are shown be trained efficiently by our method. As a simple direct search method, it can be applied in hardware implementations easily.

With the advantages of simple structure, high electromagnetic (EM) shielding effectiveness (SE), wide shielding bandwidth and adjustable transmittance, the metal grid is a popular tool in the field of electromagnetic shielding. In this work, the deep learning algorithm of artificial neural network was used to predict the EM SE of the metal grid. After certain number of trainings, both the SE spectrum and the minimum SE value in a given frequency band can be predicted accurately and effectively. On the other hand, for a target SE, the design parameters of the metal grid structure can be immediately predicted. The results show that the relative mean square deviation of the predicted SE curve is about 5%, the relative error of the minimum SE value of the curve is less than 5%, and the average relative error of the minimum SE value in the predicted frequency band is 11.5%.