- Research Article
42
- 10.1016/s0893-6080(96)00089-5
On Fault Tolerant Training of Feedforward Neural Networks
- Apr 01, 1997
- Neural Networks
- Behnam S Arad + 1 more +1
On Fault Tolerant Training of Feedforward Neural Networks
This paper presents an application of the scaled Givens rotations in the process of feedforward artificial neural networks training. This method bases on the QR decomposition. The paper describes mathematical background that needs to be considered during the application of the scaled Givens rotations in neural networks training. The paper concludes with sample simulation results.
On Fault Tolerant Training of Feedforward Neural Networks
On Fault Tolerant Training of Feedforward Neural Networks
On Performance of Marine Predators Algorithm in Training of Feed-Forward Neural Network for Identification of Nonlinear Systems
Artificial neural networks (ANNs) are used to solve many problems, such as modeling, identification, prediction, and classification. The success of ANN is directly related to the training process. Meta-heuristic algorithms are used extensively for ANN training. Within the scope of this study, a feed-forward artificial neural network (FFNN) is trained using the marine predators algorithm (MPA), one of the current meta-heuristic algorithms. Namely, this study is aimed to evaluate the performance of MPA in ANN training in detail. Identification/modeling of nonlinear systems is chosen as the problem. Six nonlinear systems are used in the applications. Some of them are static, and some are dynamic. Mean squared error (MSE) is utilized as the error metric. Effective training and testing results were obtained using MPA. The best mean error values obtained for six nonlinear systems are 2.3 × 10−4, 1.8 × 10−3, 1.0 × 10−4, 1.0 × 10−4, 1.2 × 10−5, and 2.5 × 10−4. The performance of MPA is compared with 16 meta-heuristic algorithms. The results have shown that the performance of MPA is better than other algorithms in ANN training for the identification of nonlinear systems.
Read moreTraining of Feed-Forward Neural Networks by Using Optimization Algorithms Based on Swarm-Intelligent for Maximum Power Point Tracking
One of the most used artificial intelligence techniques for maximum power point tracking is artificial neural networks. In order to achieve successful results in maximum power point tracking, the training process of artificial neural networks is important. Metaheuristic algorithms are used extensively in the literature for neural network training. An important group of metaheuristic algorithms is swarm-intelligent-based optimization algorithms. In this study, feed-forward neural network training is carried out for maximum power point tracking by using 13 swarm-intelligent-based optimization algorithms. These algorithms are artificial bee colony, butterfly optimization, cuckoo search, chicken swarm optimization, dragonfly algorithm, firefly algorithm, grasshopper optimization algorithm, krill herd algorithm, particle swarm optimization, salp swarm algorithm, selfish herd optimizer, tunicate swarm algorithm, and tuna swarm optimization. Mean squared error is used as the error metric, and the performances of the algorithms in different network structures are evaluated. Considering the results, a success ranking score is obtained for each algorithm. The three most successful algorithms in both training and testing processes are the firefly algorithm, selfish herd optimizer, and grasshopper optimization algorithm, respectively. The training error values obtained with these algorithms are 4.5 × 10−4, 1.6 × 10−3, and 2.3 × 10−3, respectively. The test error values are 4.6 × 10−4, 1.6 × 10−3, and 2.4 × 10−3, respectively. With these algorithms, effective results have been achieved in a low number of evaluations. In addition to these three algorithms, other algorithms have also achieved mostly acceptable results. This shows that the related algorithms are generally successful ANFIS training algorithms for maximum power point tracking.
Read moreA review on neural networks with random weights
A review on neural networks with random weights
Harmony Search Based Supervised Training of Artificial Neural Networks
This paper presents a novel technique for the supervised training of feed-forward artificial neural networks (ANN) using the Harmony Search (HS) algorithm. HS is a stochastic meta-heuristic that is inspired from the improvisation process of musicians. Unlike Backpropagation, HS is non-trajectory driven. By modifying an existing improved version of HS & adopting a suitable ANN data representation, we propose a training technique where two of HS probabilistic parameters are determined dynamically based on the best-to-worst (BtW) harmony ratio in the current harmony memory instead of the improvisation count. This would be more suitable for ANN training since parameters and termination would depend on the quality of the attained solution. We have empirically tested and verified our technique by training an ANN with a benchmarking problem. In terms of overall training time and recognition, our results have revealed that our method is superior to both the original improved HS and standard Backpropagation.
Read moreA new evolutionary neural networks based on intrusion detection systems using locust swarm optimization
The need to avoid computer system breaches is increasing. Many researchers have adopted different approaches, such as intrusion detection systems (IDSs), to handle various threats. Intrusion detection has become an imperative system to detect various security breaches. Until today, researchers face the problem of building reliable and effective IDSs that can handle numerous attacks with changing patterns. This paper deals with feed-forward neural network (FNN) training problems using the application of a recently invented meta-heuristic optimization algorithm locust swarm optimization (LSO) for the first time. FNN is combined with LSO (FNN-LSO) to build an advanced detection system and improve the performance of IDS. Our method is applied to a series of experiments to study the capability and performance of the proposed approach. Experimental studies began by using intrusion detection evaluation data, namely, NSL-KDD and UNSW-NB15, to benchmark the performance of the proposed approach. The most common evolutionary trainers, namely, particle swarm optimizer PSO-based trainer and genetic algorithm GA-based trainer, were implemented to verify the results. Compared with existing methods in the literature, our proposed approach provides to be more accurate to be an alternative solution for IDS. The experimental results show that our training algorithm not only attained a very good performance in terms of speed convergence but also achieved reliability due to the reduced likelihood of being trapped in local minima. Furthermore, our proposed model improves the detection rate.
Read moreTraining of Artificial Neural Network Using PSO With Novel Initialization Technique
Artificial neural networks (ANN) have been widely used in the field of data classification. Normally, training of neural network is applied with the traditional back propagation technique. As, this approach has various drawbacks, training of neural network is done with Particle Swarm Optimization (PSO). PSO has been widely used to solve the diverse kind of optimization problems. Population initialization performs a significant role in meta-heuristic algorithms. This paper describes a new initialization population approach Log Logistic termed as PSOLL-NN to create the initialization of the swarm. The proposed algorithm has been tested for weight optimization of feed forward neural network; and compared with back propagation Algorithm (BPA), standard PSO (PSO-NN), PSO initialized with Halton Sequence (PSOH-NN), Torus sequence (PSOT-NN) and Sobol sequence (PSOS-NN). The experimental results show that the proposed technique performed exceptionally better than the other traditional techniques. Moreover, the outcome of our work presents a foresight that how the proposed initialization technique can be used as an efficient alternative to standard training approaches for the data classification problems.
Read moreGuided convergence for training feed-forward neural network using novel gravitational search optimization
Training of feed-forward neural network using stochastic optimizationtechniques recently gained a lot of importance invarious pattern recognition and data miningapplications because of its capability of escaping local minima trap. However such techniques may suffer fromslow and poor convergence. This fact inspires us to work onmeta-heuristic optimization technique for training the neural network. In this respect, to train the neural network, we focus on implementing thegravitational search algorithm(GSA) which is based on the Newton's law of motion principle and the interaction of masses. GSA has good ability to search for the global optimum, but it may suffer from slow searching speed in the lastiterations. Our work is directed towards the smart convergence by modifying the original GSA and also guiding the algorithm to make it immune to local minima trap. Results on various benchmark datasets prove the robustness of the modified algorithm.
Read moreArtificial neural networks broken rotor bars induction motor fault detection
Paper deals with application of online rotor broken bar fault detection via artificial neural networks. Fault can be detected by monitoring abnormalities of the spectrum amplitudes at certain frequencies in the motor current spectrum. These discriminative features are used for training of feed-forward backpropagation artificial neural network. Trained network is capable to successfully classify induction motor rotor condition. Results are presented in tables and figures.
Read moreTowards a Very Fast Feedforward Multilayer Neural Networks Training Algorithm
** This paper presents a novel fast algorithm for feedforward neural networks training. It is based on the Recursive Least Squares (RLS) method commonly used for designing adaptive filters. Besides, it utilizes two techniques of linear algebra, namely the orthogonal transformation method, called the Givens Rotations (GR), and the QR decomposition, creating the GQR (symbolically we write GR + QR = GQR) procedure for solving the normal equations in the weight update process. In this paper, a novel approach to the GQR algorithm is presented. The main idea revolves around reducing the computational cost of a single rotation by eliminating the square root calculation and reducing the number of multiplications. The proposed modification is based on the scaled version of the Givens rotations, denoted as SGQR. This modification is expected to bring a significant training time reduction comparing to the classic GQR algorithm. The paper begins with the introduction and the classic Givens rotation description. Then, the scaled rotation and its usage in the QR decomposition is discussed. The main section of the article presents the neural network training algorithm which utilizes scaled Givens rotations and QR decomposition in the weight update process. Next, the experiment results of the proposed algorithm are presented and discussed. The experiment utilizes several benchmarks combined with neural networks of various topologies. It is shown that the proposed algorithm outperforms several other commonly used methods, including well known Adam optimizer.
Read moreTraining Feed-Forward Neural Networks Employing Improved Bat Algorithm for Digital Image Compression
Training of feed-forward neural networks is a well-known and a vital optimization problem which is used to digital image lossy compression. Since the inter-pixel relationship in the picture is highly non-linear and unpredictive in the absence of a prior knowledge of the picture itself, it has shown that the neural networks combined with metaheuristics can be very efficient optimization method for image compression issues. In this paper, we propose an improved bat algorithm for training the input-output weights of the network which contains input-output layers of the equal sizes and a hidden layer of smaller size in-between. It has applied on five standard digital images. From the experimental analysis, it can be shown that the proposed method produces an acceptable quality of the compressed image as well as a good ratio of compression.
Read moreThe Training of Pi-Sigma Artificial Neural Networks with Differential Evolution Algorithm for Forecasting
Looking at the artificial neural networks’ literature, most of the studies started with feedforward artificial neural networks and the training of many feedforward artificial neural networks models are performed with derivative-based algorithms such as levenberg–marquardt and back-propagation learning algorithms in the first studies. In recent years, although many new heuristic algorithms have been proposed for different aims these heuristic algorithms are also frequently used in the training process of many different artificial neural network models. Pi-sigma artificial neural networks have different importance than many artificial neural network models with its higher-order network structure and superior forecasting performance. In this study, the training of Pi-Sigma artificial neural networks is performed by differential evolution algorithm uses DE/rand/1 mutation strategy. The performance of the proposed method is evaluated by two data sets and seen that the proposed method has a very effective performance compared with many artificial neural network models.
Read moreToward the Training of Feed-Forward Neural Networks With the D-Optimum Input Sequence
The problem under consideration is to obtain a measurement schedule for training neural networks. This task is perceived as an experimental design in a given design space that is obtained in such a way as to minimize the difference between the neural network and the system being considered. This difference can be expressed in many different ways and one of them, namely, the D-optimality criterion is used in this paper. In particular, the paper presents a unified and comprehensive treatment of this problem by discussing the existing and previously unpublished properties of the optimum experimental design (OED) for neural networks. The consequences of the above properties are discussed as well. A hybrid algorithm that can be used for both the training and data development of neural networks is another important contribution of this paper. A careful analysis of the algorithm is presented and its comprehensive convergence analysis with the help of the Lyapunov method are given. The paper contains a number of numerical examples that justify the application of the OED theory for neural networks. Moreover, an industrial application example is given that deals with the valve actuator.
Read moreA parallel algorithm for gradient training of feedforward neural networks
A parallel algorithm for gradient training of feedforward neural networks
Electricity Demand Forecasting Based on Feedforward Neural Network Training by a Novel Hybrid Evolutionary Algorithm
Electricity demand forecasting is an important index to make power development plan and dispatch the loading of generating units in order to meet system demand. In order to improve the accuracy of the forecasting, we apply the feedforward neural network for electricity demand forecasting. Inspired by the idea of Artificial Fish Swarm Algorithm, in this paper we proposed one hybrid evolutionary algorithm which based on PSO and AFSA methods through crossing over the PSO and AFSA algorithms to train the feedforward neural network. This proposed method has been applied in a real electricity load forecasting, the results show that the proposed approach has a better generalization performance and is also more accurate and effective than the feedforward neural network trained by particle swarm optimization.
Read more