Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and BP Algorithms

In applying neural network to identification, back propagation (BP) algorithm is usually trapped to a local optimum and has a low speed of convergence, whereas particle swarm optimization (PSO) is advantageous in terms of global optimal searching. In this paper, a new algorithms which combines guaranteed convergence particle swarm optimizer (GCPSO) algorithm with BP (Back Propagation) algorithm is introduced in this paper and applied to BP neural networks to optimize the parameters of BP neural networks so as to improve the convergence speed and precision of BP neural networks. Compared with the BP algorithm and PSO-BP algorithm, the results of the simulation show that the algorithm of BP Neural Network Model Based on Guaranteed Convergence Particle Swarm Optimization speeds up the convergence process and enhances the accurate rate in pattern recognition.

This paper proposed Global Artificial Bee Colony algorithm for training Neural Network (NN), which is a globalised form of standard Artificial Bee Colony algorithm. NN trained with the standard backpropagation (BP) algorithm normally utilizes computationally intensive training algorithms. One of the crucial problems with the BP algorithm is that it can sometimes yield the networks with suboptimal weights because of the presence of many local optima in the solution space. To overcome, GABC algorithm used in this work to train MLP learning for classification problem, the performance of GABC is benchmarked against MLP training with the typical BP, ABC and Particle swarm optimization for boolean function classification. The experimental result shows that MLP-GABC performs better than that standard BP, ABC and PSO for the classification task.

Accelerating neural network training using weight extrapolations

Stability analysis of a three-term backpropagation algorithm

To address the insufficient expressive capabilities of traditional methods in assessing the development level of rural tourism, this study explores the fusion application of the Bidirectional Encoder Representations from Transformers (BERT) and the Back Propagation (BP) algorithm to enhance the accuracy and comprehensiveness of rural tourism development assessment. Firstly, this study introduces the BERT deep learning model and its applications in natural language processing, alongside the role of the BP algorithm in pattern recognition and predictive analysis. Subsequently, a framework for assessing rural tourism development levels, integrating BERT and the BP algorithm, is proposed. This framework collects multidimensional rural tourism-related data and utilizes the BERT model for sentiment analysis and topic extraction from textual data. Empirical analysis of rural tourism development in a specific region validates the effectiveness of the proposed approach. Experimental results demonstrate: (1) The model achieves an accuracy of 84.33% and an F1 score of 85.33% on the publicly available Laptop dataset, with a processing time of 20 s, significantly outperforming other methods. Compared to traditional approaches, the proposed method accurately captures correlations between textual information and numerical data, thereby enhancing the credibility and accuracy of assessment results. (2) From the ablation study results, it is evident that removing any component from the model leads to performance degradation. Specifically, removing the Bidirectional Gated Recurrent Unit (BiGRU) reduces accuracy and F1 scores to 78.21% and 76.33% on the Laptop dataset, and to 85.10% and 70.45% on the Tourist_F dataset. Removing Text Convolutional Neural Network (CNN) reduces accuracy and F1 scores to 79.34% and 77.56% on the Laptop dataset, and to 86.25% and 72.11% on the Tourist_F dataset. The most significant performance decline occurs upon removing BERT, with accuracy and F1 scores decreasing to 70.14% and 66.43% on the Laptop dataset, and to 78.45% and 60.33% on the Tourist_F dataset. These results indicate that BiGcRU, TextCNN, and BERT each contribute significantly to the model’s performance. This study provides substantial support for advancing sustainable development in rural tourism, offering significant practical and innovative value.

The back propagation (BP) algorithm is the most popular supervised learning method for multi-layered feed forward Neural Network. It has been successfully deployed in numerous practical problems and disciplines. Regardless of its popularity, BP is still known for some major drawbacks such as easily getting stuck in local minima and slow convergence; since, it uses Gradient Descent (GD) method to learn the network. Over the years, many improved modifications of the BP learning algorithm have been made by researchers but the local minima problem remains unresolved. Therefore, to resolve the inherent problems of BP algorithm, this paper proposed BPGD-A3T algorithm where the approach introduces three adaptive parameters which are gain, momentum and learning rate in BP. The performance of the proposed BPGD-A3T algorithm is then compared with BPGD two term parameters (BPGD-2T), BP with adaptive gain (BPGD-AG) and conventional BP algorithm (BPGD) by means of simulations on classification datasets. The simulation results show that the proposed BPGD-A3T shows better performance and performed highest accuracy for all dataset as compared to other.

Instantaneous learning is a desirable feature in neural networks. This type of learning enables the network to be trained very quickly, typically in just one or two passes of the training set as opposed to hundreds or even thousands of passes for networks trained by an iterative process, such as the error backpropagation (BP) algorithm. This dissertation first reviews several existing types of neural networks with instantaneous learning capability. It then proposes a new network called the Fuzzy Classification (FC) Neural Network that can be trained with two passes of the training samples. The first pass assigns the synaptic weights for the input and output layers. The second pass determines the radius of generalization r for each training sample. The network exhibits fuzziness in two regards: (1) by fuzzification of the location of each training vector in the input space; and (2) by assigning fuzzy memberships of output classes to new input vectors. The operation of the FC network is analyzed from different perspectives, namely, separability of patterns, curve fitting and kernel regression. It is shown that one mode of operation of the FC network can be made to behave like a Radial Basis Function (RBF) network by proper choice of membership function and network parameter. The performance of the FC network as a pattern classifier is also discussed in a statistical framework. The generalization performance of the FC network is compared against that of the CC4 Comer Classification Neural Network, the RBF network, and the Multilayer Perceptron network trained by the BP algorithm. Experiments involving a Henon map time series, a Mackey-Glass time series, and a spiral pattern are used in the comparison. The performance of the FC network is found to be comparable to that of RBF and BP networks, and better than that of the CC4 network. Its performance is also more scalable than those of CC4 and RBF networks. Further, the time taken to design an FC network for any given problem is much shorter compared to the other three networks.

The assembly line balancing problem deals with the assignment of tasks to work stations. Mixed-model assembly line problem is a type of assembly line balancing problem at which two or more models of the same product are assembled sequentially at the same line. To achieve optimality and efficiency of solving this problem, tasks at each work station have to be well balanced satisfying all constraints. This paper deals with the mixed-model assembly line balancing problem (MALBP) in which the objective is to minimize the cycle time for a given number of work stations. The problem is solved by using a hybrid of an ant colony optimization and a greedy algorithm (Ant-Greedy). MATLAB Software is used to perform the proposed method. Then, the proposed method is applied to a real case problem found in the literature for the assembly line of automatic changeover in the Electronic Industries Company in Iraq. The results of the proposed method are compared with the performance of the Merging Shortest and Longest Operation (MMSLO) method. The comparison shows that the Ant-Greedy optimization method is more efficient, where the efficiency increased from 93.53% for MMSLO method to 97.26% for the Ant-Greedy method.

Back Propagation (BP) algorithm is one of the oldest learning techniques used by artificial neural networks (ANN). It has successfully been implemented in various practical problems. However, the algorithm still faces some drawbacks such as getting easily stuck at local minima and needs longer time to converge on an acceptable solution. Recently, the introduction of Second Order Methods has shown a significant improvement on the learning in BP but it still has some drawbacks such as slow convergence and complexity. To overcome these limitations, this research proposed a modified approach for BP by introducing the Conjugate Gradient and Quasi-Newton which were Second Order methods together with ‘gain’ parameter. The performances of the proposed approach is evaluated in terms of lowest number of epochs, lowest CPU time and highest accuracy on five benchmark classification datasets such as Glass, Horse, 7Bit Parity, Indian Liver Patient and Lung Cancer. The results show that the proposed Second Order methods with ‘gain’ performed better than the BP algorithm.

A neural network using the Random Weight Change algorithm is shown able to be trained to perform on-line control of the current of an induction motor stator, despite analog circuit nonidealities. The induction motor is a complex nonlinear electromechanical system, with rapidly time-varying system parameters. Due to the small time constant of this power electronic system, the neural network must be able to finish each training cycle in less than 50 microseconds, which is only possible when controlled by specifically designed hardware circuits. An analog circuit is preferred for its ability to implement a reasonable size of network on one integrated chip. The analog circuit nonidealities are overcome by the Random Weight Change (RWC) algorithm. RWC is based on the method of random searching, and achieves similar performance to the back-propagation (BP) algorithm. The back-propagation algorithm is very difficult to implement in analog hardware due to its sensitivity to offset and nonlinearity errors. The RWC algorithm is simulated with analog circuit nonidealities, and is shown to be immune to these problems, thus the RWC algorithm is found ideally suited for the high speed analog circuit neural network implementation.

In this paper, the no-wait flow shop scheduling problem under makespan and flowtime criteria is addressed. The no-wait flowshop is a variant of the well-known flowshop scheduling problem where all processes follow the previous one without any interruption for operations of a job. Owing to the problem is known to be NP-hard for more than two machines, a hybrid meta-heuristic algorithm based on ant colony optimization (ACO) and simulated annealing (SA) algorithm is improved. First, at each step, due to the characteristic of ACO algorithm that include solution construction and pheromone trail updating, some different areas of search space are checked and best solution is selected. Then, to enhance the quality and diversity of the solution and finding best neighbor of this solution, a novel SA is presented. Moreover, a new principle is applied for global pheromone update based on the probability function like SA algorithm. The proposed approach solution is compared with several the state-of-the-art algorithms in the literature. The reported results show that the proposed algorithms are effective and the new approach for local search in ACO algorithm is efficient for solving the no-wait flow shop problem. Then, we employed another hybrid ACO algorithm based on hybridization of ACO with variable neighborhood search (VNS) and compare the results given by two proposed algorithms. These results show that our new hybrid provides better results than ACO-VNS algorithm.

Parameter estimation of fuzzy neural network controller based on a modified differential evolution

Wireless Sensor Networks (WSNs) are widely applied in various applications such as environment monitoring, precision agriculture, healthcare, and surveillance. Sensor nodes accumulate data and transmit it to a central unit or Base Station (BS). The traditional data routing mechanisms cause high energy depletion at sensor nodes. It causes the premature death of sensor nodes. The lifetimes of sensor nodes have a direct impact on the lifetime of WSNs. To get more prolonged operation of WSNs, the lifetime of sensor nodes should be increased. This paper applies the Fuzzy C-means algorithm to create different clusters of sensor nodes. Furthermore, this paper proposes a hybrid data routing algorithm that designs an optimal route among cluster heads for data transmission. A hybrid of Ant Colony Optimization (ACO) and Bacterial Foraging Optimization (BFO) algorithm is used to identify the optimal route with in the network. The optimal data collection path minimizes the transmission distance between nodes, minimizes data collection delay and saves energy at sensor nodes. This approach has been compared with other state of the art approaches in terms of residual energy, number of alive nodes and average transmission delay. The results show that the proposed hybrid approach outperforms existing methods.

A new model based on improved ant colony algorithm (ACA) and backpropagation (BP) is proposed to predict Silicon content of hot metal in blast furnace. BP algorithm has been widely used in training artificial neural network (ANN), which is an outstanding model to predict Silicon content. BP algorithm has many attractive features, such as adaptive learning, self- organism, and fault tolerant ability. All of them make BP one of the most successful algorithms in various fields. But BP suffers from relatively slow convergence speed, extensive computations and possible divergence for certain conditions. As a new bionic algorithm, the improved ACA has gained very good performance in solving traveling salesman problem (TSP) and other optimization problems. Its properties such as distributed computation, heuristic searching and robustness have well conquered the long convergence speed and premature problem, which are the main deficiencies of BP algorithm. Experiments show the model proposed has good performance in predicting Silicon content of hot metal.

There are a number of problems associated with training neural networks with backpropagation algorithm. The algorithm scales exponentially with increased complexity of the problem. It is very often trapped in local minima, and is not robust to changes of network parameters such as number of hidden layer neurons and learning rate. The use of genetic algorithms is a recent trend, which is good at exploring a large and complex search space, to overcome such problems. In this paper a genetic algorithm is proposed for training feedforward neural networks and its performances is investigated. The results are analyzed and compared with those obtained by the backpropagation algorithm.