- Research Article
42
- 10.1016/s0893-6080(96)00089-5
On Fault Tolerant Training of Feedforward Neural Networks
- Apr 01, 1997
- Neural Networks
- Behnam S Arad + 1 more +1
On Fault Tolerant Training of Feedforward Neural Networks
The process of training feedforward neural networks (FFNNs) can benefit from an automated process where the best heuristic to train the network is sought out automatically by means of a high-level probabilistic-based heuristic. This research introduces a novel population-based Bayesian hyper-heuristic (BHH) that is used to train feedforward neural networks (FFNNs). The performance of the BHH is compared to that of ten popular low-level heuristics, each with different search behaviours. The chosen heuristic pool consists of classic gradient-based heuristics as well as meta-heuristics (MHs). The empirical process is executed on fourteen datasets consisting of classification and regression problems with varying characteristics. The BHH is shown to be able to train FFNNs well and provide an automated method for finding the best heuristic to train the FFNNs at various stages of the training process.
On Fault Tolerant Training of Feedforward Neural Networks
On Fault Tolerant Training of Feedforward Neural Networks
A review on neural networks with random weights
A review on neural networks with random weights
Feed-forward neural network training using sparse representation
Feed-forward neural network training using sparse representation
Training of Feed-Forward Neural Networks by Using Optimization Algorithms Based on Swarm-Intelligent for Maximum Power Point Tracking
One of the most used artificial intelligence techniques for maximum power point tracking is artificial neural networks. In order to achieve successful results in maximum power point tracking, the training process of artificial neural networks is important. Metaheuristic algorithms are used extensively in the literature for neural network training. An important group of metaheuristic algorithms is swarm-intelligent-based optimization algorithms. In this study, feed-forward neural network training is carried out for maximum power point tracking by using 13 swarm-intelligent-based optimization algorithms. These algorithms are artificial bee colony, butterfly optimization, cuckoo search, chicken swarm optimization, dragonfly algorithm, firefly algorithm, grasshopper optimization algorithm, krill herd algorithm, particle swarm optimization, salp swarm algorithm, selfish herd optimizer, tunicate swarm algorithm, and tuna swarm optimization. Mean squared error is used as the error metric, and the performances of the algorithms in different network structures are evaluated. Considering the results, a success ranking score is obtained for each algorithm. The three most successful algorithms in both training and testing processes are the firefly algorithm, selfish herd optimizer, and grasshopper optimization algorithm, respectively. The training error values obtained with these algorithms are 4.5 × 10−4, 1.6 × 10−3, and 2.3 × 10−3, respectively. The test error values are 4.6 × 10−4, 1.6 × 10−3, and 2.4 × 10−3, respectively. With these algorithms, effective results have been achieved in a low number of evaluations. In addition to these three algorithms, other algorithms have also achieved mostly acceptable results. This shows that the related algorithms are generally successful ANFIS training algorithms for maximum power point tracking.
Read moreElectricity Demand Forecasting Based on Feedforward Neural Network Training by a Novel Hybrid Evolutionary Algorithm
Electricity demand forecasting is an important index to make power development plan and dispatch the loading of generating units in order to meet system demand. In order to improve the accuracy of the forecasting, we apply the feedforward neural network for electricity demand forecasting. Inspired by the idea of Artificial Fish Swarm Algorithm, in this paper we proposed one hybrid evolutionary algorithm which based on PSO and AFSA methods through crossing over the PSO and AFSA algorithms to train the feedforward neural network. This proposed method has been applied in a real electricity load forecasting, the results show that the proposed approach has a better generalization performance and is also more accurate and effective than the feedforward neural network trained by particle swarm optimization.
Read moreAn evaluation of quantum neural networks in the detection of epileptic seizures in the neonatal electroencephalogram
This paper presents the results of an experimental study that evaluated the ability of quantum neural networks (QNNs) to capture and quantify uncertainty in data and compared their performance with that of conventional feedforward neural networks (FFNNs). In this work, QNNs and FFNNs were trained to classify short segments of epileptic seizures in neonatal EEG. The experiments revealed significant differences between the internal representations created by trained QNNs and FFNNs from sample information provided by the training data. The results of this experimental study also confirmed that the responses of trained QNNs are more reliable indicators of uncertainty in the input data compared with the responses of trained FFNNs.
Read moreShort-term electric power load forecasting based on cosine radial basis function neural networks: An experimental evaluation
This article presents the results of a study aimed at the development of a system for short-term electric power load forecasting. This was attempted by training feedforward neural networks (FFNNs) and cosine radial basis function (RBF) neural networks to predict future power demand based on past power load data and weather conditions. This study indicates that both neural network models exhibit comparable performance when tested on the training data but cosine RBF neural networks generalize better since they outperform considerably FFNNs when tested on the testing data. © 2005 Wiley Periodicals, Inc. Int J Int Syst 20: 591–605, 2005.
Read moreEstimation of tumor parameters using neural networks for inverse bioheat problem
Estimation of tumor parameters using neural networks for inverse bioheat problem
A fast training algorithm for extreme learning machine based on matrix decomposition
A fast training algorithm for extreme learning machine based on matrix decomposition
On Performance of Marine Predators Algorithm in Training of Feed-Forward Neural Network for Identification of Nonlinear Systems
Artificial neural networks (ANNs) are used to solve many problems, such as modeling, identification, prediction, and classification. The success of ANN is directly related to the training process. Meta-heuristic algorithms are used extensively for ANN training. Within the scope of this study, a feed-forward artificial neural network (FFNN) is trained using the marine predators algorithm (MPA), one of the current meta-heuristic algorithms. Namely, this study is aimed to evaluate the performance of MPA in ANN training in detail. Identification/modeling of nonlinear systems is chosen as the problem. Six nonlinear systems are used in the applications. Some of them are static, and some are dynamic. Mean squared error (MSE) is utilized as the error metric. Effective training and testing results were obtained using MPA. The best mean error values obtained for six nonlinear systems are 2.3 × 10−4, 1.8 × 10−3, 1.0 × 10−4, 1.0 × 10−4, 1.2 × 10−5, and 2.5 × 10−4. The performance of MPA is compared with 16 meta-heuristic algorithms. The results have shown that the performance of MPA is better than other algorithms in ANN training for the identification of nonlinear systems.
Read moreGuided convergence for training feed-forward neural network using novel gravitational search optimization
Training of feed-forward neural network using stochastic optimizationtechniques recently gained a lot of importance invarious pattern recognition and data miningapplications because of its capability of escaping local minima trap. However such techniques may suffer fromslow and poor convergence. This fact inspires us to work onmeta-heuristic optimization technique for training the neural network. In this respect, to train the neural network, we focus on implementing thegravitational search algorithm(GSA) which is based on the Newton's law of motion principle and the interaction of masses. GSA has good ability to search for the global optimum, but it may suffer from slow searching speed in the lastiterations. Our work is directed towards the smart convergence by modifying the original GSA and also guiding the algorithm to make it immune to local minima trap. Results on various benchmark datasets prove the robustness of the modified algorithm.
Read moreData-driven Prediction of Temperature Evolution in Metallic Additive Manufacturing Process
In this study, a data-driven deep learning model for fast and accurate prediction of temperature evolution and melting pool size of metallic additive manufacturing processes are developed. The study focuses on bulk experiments of the M4 high-speed steel material powder manufactured by Direct Energy Deposition. Under non-optimized process parameters, many deposited layers (above 30) generate large changes of microstructure through the sample depth caused by the high sensitivity of the cladding material on the thermal history. A 2D finite element analysis (FEA) of the bulk sample, validated in a previous study by experimental measurements, is able to achieve numerical data defining the temperature field evolution under different process settings. A Feed-forward neural networks (FFNN) approach is trained to reproduce the temperature fields generated from FEA. Hence, the trained FFNN is used to predict the history of the temperature fields for new process parameter sets not included in the initial dataset. Besides the input energy, nodal coordinates, and time, five additional features relating layer number, laser location, and distance from the laser to sampling point are considered to enhance prediction accuracy. The results indicate that the temperature evolution is predicted well by the FFNN with an accuracy of 99% within 12 seconds.
Read moreTraining Feed-Forward Neural Networks Employing Improved Bat Algorithm for Digital Image Compression
Training of feed-forward neural networks is a well-known and a vital optimization problem which is used to digital image lossy compression. Since the inter-pixel relationship in the picture is highly non-linear and unpredictive in the absence of a prior knowledge of the picture itself, it has shown that the neural networks combined with metaheuristics can be very efficient optimization method for image compression issues. In this paper, we propose an improved bat algorithm for training the input-output weights of the network which contains input-output layers of the equal sizes and a hidden layer of smaller size in-between. It has applied on five standard digital images. From the experimental analysis, it can be shown that the proposed method produces an acceptable quality of the compressed image as well as a good ratio of compression.
Read moreA New Variant of the GQR Algorithm for Feedforward Neural Networks Training
This paper presents an application of the scaled Givens rotations in the process of feedforward artificial neural networks training. This method bases on the QR decomposition. The paper describes mathematical background that needs to be considered during the application of the scaled Givens rotations in neural networks training. The paper concludes with sample simulation results.
Read moreEnhanced Salp Swarm Algorithm based on random walk and its application to training feedforward neural networks
Salp Swarm Algorithm (SSA) is a new type of metaheuristic and has shown superiority over other well-known algorithms such as Particle Swarm Optimization and Grey Wolf Optimizer in solving challenging optimization problems. Despite its superior performance, SSA still has problems such as insufficient convergence speed. Moreover, its local optima avoidance ability is not as good as those evolutionary algorithms using crossover operators. In this paper, we propose a modified Salp Swarm Algorithm (m-SSA) which improves the exploitation and exploration of SSA by integrating random walk strategy and especially enhances exploration by adding a new controlling parameter. In addition, a simulated annealing-type acceptance criterion is adopted to accept the fittest follower position as the new best leader position. The performance of the proposed algorithm is benchmarked on a set of classical functions and CEC2014 test suite. The proposed algorithm (m-SSA) outperforms SSA significantly on most test functions. When compared with other state-of-the-art metaheuristics, it also presents very competitive results. Besides, we apply the proposed algorithm on training feedforward neural networks (FNNs) and the results prove the effectiveness and efficiency of m-SSA.
Read more