- Research Article
42
- 10.1016/s0893-6080(96)00089-5
On Fault Tolerant Training of Feedforward Neural Networks
- Apr 01, 1997
- Neural Networks
- Behnam S Arad + 1 more +1
On Fault Tolerant Training of Feedforward Neural Networks
Training of feed-forward neural networks is a well-known and a vital optimization problem which is used to digital image lossy compression. Since the inter-pixel relationship in the picture is highly non-linear and unpredictive in the absence of a prior knowledge of the picture itself, it has shown that the neural networks combined with metaheuristics can be very efficient optimization method for image compression issues. In this paper, we propose an improved bat algorithm for training the input-output weights of the network which contains input-output layers of the equal sizes and a hidden layer of smaller size in-between. It has applied on five standard digital images. From the experimental analysis, it can be shown that the proposed method produces an acceptable quality of the compressed image as well as a good ratio of compression.
On Fault Tolerant Training of Feedforward Neural Networks
On Fault Tolerant Training of Feedforward Neural Networks
A review on neural networks with random weights
A review on neural networks with random weights
Training of Feed-Forward Neural Networks by Using Optimization Algorithms Based on Swarm-Intelligent for Maximum Power Point Tracking
One of the most used artificial intelligence techniques for maximum power point tracking is artificial neural networks. In order to achieve successful results in maximum power point tracking, the training process of artificial neural networks is important. Metaheuristic algorithms are used extensively in the literature for neural network training. An important group of metaheuristic algorithms is swarm-intelligent-based optimization algorithms. In this study, feed-forward neural network training is carried out for maximum power point tracking by using 13 swarm-intelligent-based optimization algorithms. These algorithms are artificial bee colony, butterfly optimization, cuckoo search, chicken swarm optimization, dragonfly algorithm, firefly algorithm, grasshopper optimization algorithm, krill herd algorithm, particle swarm optimization, salp swarm algorithm, selfish herd optimizer, tunicate swarm algorithm, and tuna swarm optimization. Mean squared error is used as the error metric, and the performances of the algorithms in different network structures are evaluated. Considering the results, a success ranking score is obtained for each algorithm. The three most successful algorithms in both training and testing processes are the firefly algorithm, selfish herd optimizer, and grasshopper optimization algorithm, respectively. The training error values obtained with these algorithms are 4.5 × 10−4, 1.6 × 10−3, and 2.3 × 10−3, respectively. The test error values are 4.6 × 10−4, 1.6 × 10−3, and 2.4 × 10−3, respectively. With these algorithms, effective results have been achieved in a low number of evaluations. In addition to these three algorithms, other algorithms have also achieved mostly acceptable results. This shows that the related algorithms are generally successful ANFIS training algorithms for maximum power point tracking.
Read moreGuided convergence for training feed-forward neural network using novel gravitational search optimization
Training of feed-forward neural network using stochastic optimizationtechniques recently gained a lot of importance invarious pattern recognition and data miningapplications because of its capability of escaping local minima trap. However such techniques may suffer fromslow and poor convergence. This fact inspires us to work onmeta-heuristic optimization technique for training the neural network. In this respect, to train the neural network, we focus on implementing thegravitational search algorithm(GSA) which is based on the Newton's law of motion principle and the interaction of masses. GSA has good ability to search for the global optimum, but it may suffer from slow searching speed in the lastiterations. Our work is directed towards the smart convergence by modifying the original GSA and also guiding the algorithm to make it immune to local minima trap. Results on various benchmark datasets prove the robustness of the modified algorithm.
Read moreToward the Training of Feed-Forward Neural Networks With the D-Optimum Input Sequence
The problem under consideration is to obtain a measurement schedule for training neural networks. This task is perceived as an experimental design in a given design space that is obtained in such a way as to minimize the difference between the neural network and the system being considered. This difference can be expressed in many different ways and one of them, namely, the D-optimality criterion is used in this paper. In particular, the paper presents a unified and comprehensive treatment of this problem by discussing the existing and previously unpublished properties of the optimum experimental design (OED) for neural networks. The consequences of the above properties are discussed as well. A hybrid algorithm that can be used for both the training and data development of neural networks is another important contribution of this paper. A careful analysis of the algorithm is presented and its comprehensive convergence analysis with the help of the Lyapunov method are given. The paper contains a number of numerical examples that justify the application of the OED theory for neural networks. Moreover, an industrial application example is given that deals with the valve actuator.
Read moreOn Performance of Marine Predators Algorithm in Training of Feed-Forward Neural Network for Identification of Nonlinear Systems
Artificial neural networks (ANNs) are used to solve many problems, such as modeling, identification, prediction, and classification. The success of ANN is directly related to the training process. Meta-heuristic algorithms are used extensively for ANN training. Within the scope of this study, a feed-forward artificial neural network (FFNN) is trained using the marine predators algorithm (MPA), one of the current meta-heuristic algorithms. Namely, this study is aimed to evaluate the performance of MPA in ANN training in detail. Identification/modeling of nonlinear systems is chosen as the problem. Six nonlinear systems are used in the applications. Some of them are static, and some are dynamic. Mean squared error (MSE) is utilized as the error metric. Effective training and testing results were obtained using MPA. The best mean error values obtained for six nonlinear systems are 2.3 × 10−4, 1.8 × 10−3, 1.0 × 10−4, 1.0 × 10−4, 1.2 × 10−5, and 2.5 × 10−4. The performance of MPA is compared with 16 meta-heuristic algorithms. The results have shown that the performance of MPA is better than other algorithms in ANN training for the identification of nonlinear systems.
Read moreReducing Feed-Forward Neural Network Processing Time Utilizing Matrix Multiplication Algorithms on Heterogeneous Distributed Systems
This paper presents a work in progress that aims to reduce the overall training and processing time of feed-forward multi-layer neural networks. If the network is large processing is expensive in terms of both; time and space. In this paper, we suggest a cost-effective and presumably a faster processing technique by utilizing a heterogeneous distributed system composed of a set of commodity computers connected by a local area network. Neural network computations can be viewed as a set of matrix multiplication processes. These can be adapted to utilize the existing matrix multiplication algorithms tailored for such systems. With Java technology as an implementation means, we discuss the different factors that should be considered in order to achieve this goal highlighting some issues that might affect such a proposed implementation.
Read moreA new evolutionary neural networks based on intrusion detection systems using locust swarm optimization
The need to avoid computer system breaches is increasing. Many researchers have adopted different approaches, such as intrusion detection systems (IDSs), to handle various threats. Intrusion detection has become an imperative system to detect various security breaches. Until today, researchers face the problem of building reliable and effective IDSs that can handle numerous attacks with changing patterns. This paper deals with feed-forward neural network (FNN) training problems using the application of a recently invented meta-heuristic optimization algorithm locust swarm optimization (LSO) for the first time. FNN is combined with LSO (FNN-LSO) to build an advanced detection system and improve the performance of IDS. Our method is applied to a series of experiments to study the capability and performance of the proposed approach. Experimental studies began by using intrusion detection evaluation data, namely, NSL-KDD and UNSW-NB15, to benchmark the performance of the proposed approach. The most common evolutionary trainers, namely, particle swarm optimizer PSO-based trainer and genetic algorithm GA-based trainer, were implemented to verify the results. Compared with existing methods in the literature, our proposed approach provides to be more accurate to be an alternative solution for IDS. The experimental results show that our training algorithm not only attained a very good performance in terms of speed convergence but also achieved reliability due to the reduced likelihood of being trapped in local minima. Furthermore, our proposed model improves the detection rate.
Read moreElectricity Demand Forecasting Based on Feedforward Neural Network Training by a Novel Hybrid Evolutionary Algorithm
Electricity demand forecasting is an important index to make power development plan and dispatch the loading of generating units in order to meet system demand. In order to improve the accuracy of the forecasting, we apply the feedforward neural network for electricity demand forecasting. Inspired by the idea of Artificial Fish Swarm Algorithm, in this paper we proposed one hybrid evolutionary algorithm which based on PSO and AFSA methods through crossing over the PSO and AFSA algorithms to train the feedforward neural network. This proposed method has been applied in a real electricity load forecasting, the results show that the proposed approach has a better generalization performance and is also more accurate and effective than the feedforward neural network trained by particle swarm optimization.
Read moreA New Variant of the GQR Algorithm for Feedforward Neural Networks Training
This paper presents an application of the scaled Givens rotations in the process of feedforward artificial neural networks training. This method bases on the QR decomposition. The paper describes mathematical background that needs to be considered during the application of the scaled Givens rotations in neural networks training. The paper concludes with sample simulation results.
Read moreA back-propagation neural network based on a hybrid genetic algorithm and particle swarm optimization for image compression
In this paper, an improved approach integrating genetic algorithm and adaptive particle swarm optimization with feed forward neural networks for image compression is proposed. The hybrid genetic algorithm with a novel mutation strategy and particle swarm optimization is used to train the neural network to near global optimum weights and thresholds at first. Then the network is trained with gradient descending learning algorithm to obtain the optimal network parameters. Then, the trained network is applied to the image compression. Results show that at the same compression rate the application of optimized neural network in image compression will achieve better image quality compared with the application of traditional neural network.
Read moreShort-term electric power load forecasting based on cosine radial basis function neural networks: An experimental evaluation
This article presents the results of a study aimed at the development of a system for short-term electric power load forecasting. This was attempted by training feedforward neural networks (FFNNs) and cosine radial basis function (RBF) neural networks to predict future power demand based on past power load data and weather conditions. This study indicates that both neural network models exhibit comparable performance when tested on the training data but cosine RBF neural networks generalize better since they outperform considerably FFNNs when tested on the testing data. © 2005 Wiley Periodicals, Inc. Int J Int Syst 20: 591–605, 2005.
Read moreAn evaluation of quantum neural networks in the detection of epileptic seizures in the neonatal electroencephalogram
This paper presents the results of an experimental study that evaluated the ability of quantum neural networks (QNNs) to capture and quantify uncertainty in data and compared their performance with that of conventional feedforward neural networks (FFNNs). In this work, QNNs and FFNNs were trained to classify short segments of epileptic seizures in neonatal EEG. The experiments revealed significant differences between the internal representations created by trained QNNs and FFNNs from sample information provided by the training data. The results of this experimental study also confirmed that the responses of trained QNNs are more reliable indicators of uncertainty in the input data compared with the responses of trained FFNNs.
Read moreHarmony Search Based Supervised Training of Artificial Neural Networks
This paper presents a novel technique for the supervised training of feed-forward artificial neural networks (ANN) using the Harmony Search (HS) algorithm. HS is a stochastic meta-heuristic that is inspired from the improvisation process of musicians. Unlike Backpropagation, HS is non-trajectory driven. By modifying an existing improved version of HS & adopting a suitable ANN data representation, we propose a training technique where two of HS probabilistic parameters are determined dynamically based on the best-to-worst (BtW) harmony ratio in the current harmony memory instead of the improvisation count. This would be more suitable for ANN training since parameters and termination would depend on the quality of the attained solution. We have empirically tested and verified our technique by training an ANN with a benchmarking problem. In terms of overall training time and recognition, our results have revealed that our method is superior to both the original improved HS and standard Backpropagation.
Read moreA parallel algorithm for gradient training of feedforward neural networks
A parallel algorithm for gradient training of feedforward neural networks