A neurocomputing model for real coded genetic algorithm with the minimal generation gap

This paper studies the frequency assignment problem (FAP) in TD-SCDMA network of mobile communications industry in China. The problem considers finding the optimal frequency allocation scheme for carriers with a limited frequency resource, such that the entire network interference is minimized. Besides, the allocation of frequencies needs to satisfy some constraints to avoid the effect of call interference within the same cell or adjacent cell. Given the formula for calculation of the network interference, we take the FAP as a constrained optimization problem and use a hyper-heuristic genetic algorithm (HHGA) to optimize the assignment of frequencies. We first define six low-level heuristics (LLHs) search strategies based on the computation of interference, and then use genetic algorithm (GA) at a high-level to find the best combination sequence of LLH strategies to reduce interferences of the overall network. GA uses two-point crossover, uniform mutation, and Minimal Generation Gap (MGG) as the generation alternation model. In order to speed up the search, we define a Tabu table to avoid repeat search of LLHs. Compared with scatter search as one of the meta-heuristic algorithm with best performance, our experimental results on real data sets of TD-SCDMA network have shown better result.

Spiking neural networks (SNNs) have been widely used due to their strong biological interpretability and high-energy efficiency. With the introduction of the backpropagation algorithm and surrogate gradient, the structure of SNNs has become more complex, and the performance gap with artificial neural networks (ANNs) has gradually decreased. However, most SNN hardware implementations for field-programmable gate arrays (FPGAs) cannot meet arithmetic or memory efficiency requirements, which significantly restricts the development of SNNs. They do not delve into the arithmetic operations between the binary spikes and synaptic weights or assume unlimited on-chip RAM resources using overly expensive devices on small tasks. To improve arithmetic efficiency, we analyze the neural dynamics of spiking neurons, generalize the SNN arithmetic operation to the multiplex-accumulate operation, and propose a high-performance implementation of such operation by utilizing the DSP48E2 hard block in Xilinx Ultrascale FPGAs. To improve memory efficiency, we design a memory system to enable efficient synaptic weights and membrane voltage memory access with reasonable on-chip RAM consumption. Combining the above two improvements, we propose an FPGA accelerator that can process spikes generated by the firing neurons on-the-fly (FireFly). FireFly is the first SNN accelerator that incorporates DSP optimization techniques into SNN synaptic operations. FireFly is implemented on several FPGA edge devices with limited resources but still guarantees a peak performance of 5.53 TOP/s at 300 MHz. As a lightweight accelerator, FireFly achieves the highest computational density efficiency compared with existing research using large FPGA devices.

A way in which a neural network can implement a genetic algorithm as its learning algorithm is shown. This model is called GLANN (genetic learning algorithm for neural networks). The components of GLANN can be shown to be biologically plausible. The algorithm itself can be classified as a reinforcement learning algorithm. The neural network has a fixed architecture and processes binary strings using genetic operators. Learning is stored in the form of newly created patterns, which can then be stored in some kind of associative memory. The benefits of GLANN reside in the proven optimizing capabilities of genetic algorithms, and in its parallel implementation. The shallow two-level architecture translates into system scalability, an issue that has not been successfully resolved in the case of other neural network algorithms. >

In Systems Biology, system identification, which infers regulatory network in genetic system and metabolic pathways using experimentally observed time-course data, is one of the hottest issues. The efficient numerical optimization algorithm to estimate more than 100 real-coded parameters should be developed for this purpose. New real-coded genetic algorithm (RCGA), the combination of AREX (adaptive real-coded ensemble crossover) with JGG (just generation gap), have applied to the inference of genetic interactions involving more than 100 parameters related to the interactions with using experimentally observed time-course data. Compared with conventional RCGA, the combination of UNDX (unimodal normal distribution crossover) with MGG (minimal generation gap), new algorithm has shown the superiority with improving early convergence in the first stage of search and suppressing evolutionary stagnation in the last stage of search.

The Sixth International Symposium on Neural Networks (ISNN 2009)

Shipyards have large departments or facilities. It is essential to make an effective topological layout plan since the initial investment cost of these departments is high. Topological layout is an optimization problem and Genetic Algorithm (GA) is generally used in the literature. The selection of effective genetic algorithm approaches and operators are very important to improve the performance of the optimization. This study investigates an effective solution to the shipyard topological layout using a Quadratic Assignment Problem (QAP) model with classic and elitist GA approaches. Besides, genetic operators that have significant effects on exploitation and exploration capabilities are analyzed. Therefore, 126 experiments were run with 13 different operators. The results obtained from the classic and elitist GA approach were evaluated individually and compared with each other. It was observed that the elitist GA approach has a superior performance compared to the classic GA approach. This study is the most comprehensive and practical study on the performance of the GA for topological layout of the shipyard in the literature.

In the present paper, two algorithms based on soft computing have been developed for dynamically balanced gait generations of a biped robot ascending and descending a staircase. The utility of the soft computing tools is best justified, when the data available for the problem to be solved are imprecise in nature, difficult to model and exhibit large-scale solution spaces. The problem of online gait generation of a biped robot exhibits such a complex phenomenon, and ultimately soft computing has become a natural choice for solving it. The gait generation problems of a biped robot have been solved using two different approaches, namely genetic-neural (GA-NN) and genetic-fuzzy (GA-FLC) systems. In GA-NN, the gait generation problem of a two-legged robot has been modeled using two modules of Neural Network (NN), whose weights are optimized offline using a Genetic Algorithm (GA), whereas in GA-FLC, the above problem is modeled utilizing two modules of Fuzzy Logic Controller (FLC) and their rule bases are optimized offline using a GA. Once optimized, the GA-NN and GA-FLC systems will be able to generate dynamically balanced gaits of the biped robot online. The performances of the two approaches are compared with respect to the Dynamic Balance Margin (DBM).

Low-precision training for Deep Neural Networks(DNN) has recently become a viable alternative to standard full-precision algorithms. Crucially, low-precision computation reduces both memory usage and computational cost, providing more scalability for Field Programmable Gate Arrays (FPGAs) with limited on-chip memory. In this paper, we describe and test a prototype training accelerator for Zynq All Programmable System on Chip (APSoC) devices using predominantly 8-bit integer numbers. Block floating-point quantisation and stochastic weight averaging techniques are applied during training toavoid any degradation in accuracy. Results of an implementation reveal memory savings and 17x speed-ups over processor only systems on several training tasks including the MNIST and CIFAR10 benchmarks, and online radio-frequency anomaly detection. Moreover, we propose modifications to the stochastic weight averaging low-precision (SWALP) algorithm to achieve a0.5% accuracy improvement for the abovementioned benchmarks with results within 0.1% of floating-point. We suggest that both inference and training can be deployed in the same package for stand-alone embedded applications.

Abstract: In this paper, a method for determination of refractive index in membrane of fuel cell on basis of three-longitudinal-mode laser heterodyne interferometer is presented. The optical path difference between the target and reference paths is fixed and phase shift is then calculated in terms of refractive index shift. The measurement accuracy of this system is limited by nonlinearity error. In this study, nonlinearity error is modeled by multi-layer perceptrons (MLPs) and stacked generalization method (Stacking), using two learning methods; back propagation (BP) and genetic algorithm. Training neural networks with genetic algorithm, improves modeling of nonlinearity error in this system. In the proposed technique, a real code version of genetic algorithm is used. Parameters and genetic operators are set and designed accurately. The results indicate that the nonlinearity error can be effectively modeled by training the stacking with the genetic algorithm which has minimum mean square error (MSE). Keywords: Fuel cell, Genetic algorithm, Heterodyne interferometer, Multi-layer perceptrons, Nonlinearity error, Refractive index, Stacked Generalization.

We review the recent work done by our group on theoretically analyzing the behavior of genetic algorithms(GAs) and searching for new ideas and strategies to design more efficient GAs. By introducing the concept of degree of population diversity and using the tools from finite Markov chain, we presented fairly clear characterizations of premature convergence in GAs and proved some results on the relationships between premature convergence and GA’s control parameters such as the population size, the mutation and crossover probabilities, and some population statistics. In the mean time, we have also obtained a result that depicts the search ability of the crossover operators in GAs. An upper bound on the convergence rate for the GAs has been established. Based on the theoretical analyses, we have proposed a novel GA called Genetic Algorithm with No Genetic Operators (GANGO), which needs neither to maintain a population nor to use the conventional genetic operators, and yet has the same search mechanisms as the classical GAs. As another development in improving the performance of GAs, a new concept called Dual Fitness Function was proposed which, when incorporated into the classical GAs will enable the algorithms to judge whether the current population is in the local or global attractive basins. And in this way, the control parameters of GAs can be adjusted adaptively. Some applications of our results to fields such as fuzzy systems, multi-destination routing problem (MDR) are briefly

The paper proposes a fast online learning method for neural network structures by using genetic algorithm (GA) and dynamic back propagation algorithm (BP) jointly. GA is used in the coarse tuning process which adjusts interconnection weights of the neural network. The dynamic back propagation algorithm is subsequently applied to achieve fine adjusting of the network weights. The fitness function, based on the squared error between the teaching signal and the network output value, is redefined at every time step and the proposed GA based algorithm solves a nonstationary function optimization task. At every time step the solution with the best fitness function is used for current representation of the neural network weights and biases. It is shown through the simulations and real time temperature control of drying oven that this learning algorithm has faster convergence ability and better performance on reducing mapping error in the online learning neural network structures. This leads to an improvement of the transient response of neuro adaptive systems. The proposed method has the potential to be applied to many practical areas such as system modeling and control, signal processing and pattern recognition.

In order to improve the efficiency and accuracy of the prediction of expressway traffic flow, this paper, based on the characteristics of the data of the expressway traffic flow, focuses on an optimized method of prediction with the application of the neural network with genetic algorithm. Applying genetic algorithm, optimizing BP neural network structure and establishing a new mixed model, this algorithm speed up the slow convergence velocity of traditional BP neural network prediction and increases the possibility to escape local minima. This algorithm based on the optimized genetic neural network predicts the actual data of the expressway traffic flow, the result of which shows that the application of the optimized method of prediction with the genetic neural network algorithm is effective and that it improves the rate and the accuracy of the prediction of the expressway traffic flow.

Mark Watson Morgan Kaufmann Publishers, San Francisco, 1997, 377 pp. ISBN 1-55860-420-0, $44.95 This book is well written and organized to cover how to apply AI (Artificial Intelligence) techniques for practical Internet and Intranet tools by using the Java programming language. In the book, the author uses neural network and genetic algorithms to develop distributed systems that are applied for the Internet and Intranets, such as client/server model and information retrieval tool and so on. Part 1 begins with a good general introduction to AI. The history and survey of Artificial Intelligence research are given briefly. In addition, symbolic processing versus neural networks approaches are discussed. This author opts to use neural network to develop example Internet/Intranet systems in the book. Part 2 consists of Java programs developed for AI, neural networks, genetic algorithms, natural language processing, and expert systems. Chapter 2 illustrates Graphical User Interface (GUI) class that is derived from Applet and Canvas classes in Java, which is used in most example programs in the book. Chapter 3 introduces a frame data structure that is used as a message in distributed systems in the later chapters. Chapter 4 shows neural network classes developed in Java and its theory and implementation. Chapter 5 explains the genetic algorithms and the Java genetic algorithm classes. Chapter 6 describes how to develop and implement natural language processing classes using Java. Chapter 7 shows how to share data objects in distributed systems. For example, distributed AI agent classes are developed to share data objects, WWW data collection agent classes are shown to retrieve documents in the Internet, and small POP (Post Office Protocol) mail classes are developed to send and receive mails. Chapter 8 illustrates how to develop expert systems by using Java Jess, which is based on OPS5/CLIPS expert systems. Part 3 explains how to generate and implement the projects using the Java classes that are developed in Part 2. Chapter 9 shows the project that demonstrates the use of genetic algorithms in chapter 5 to control enemy ships in a scrolling arcade game. Chapter 10 describes the real-time handwriting recognition system developed by using the neural network classes in chapter 4. Chapter 11 illustrates a greedy algorithm to optimize neural networks that is based on the author’s hand-tuning neural training data. Chapter 12 improves neural network training data by applying genetic algorithms, to discard bad training data and to avoid discarding good training data. Chapter 13 describes the distributed natural language processing system that uses natural language processing classes in chapter 6. The data structures in chapter 2 are used to store historical data in a separate server application on any other computer in the Internet or Intranets. The final chapter provides the design and implementation of a complete agent-based information retrieval tool for the Internet. The retrieved documents are stored in ASCII or HTML format. This book is a very good reference book for anyone interested in developing systems using Java in distributed environments, especially applying AI techniques. Also, the author provides the CD-ROM that includes the source code for Java classes developed in the book. Haklin Kimm, University of Tennessee

Evolving neural network using real coded genetic algorithm for daily rainfall–runoff forecasting

A hybrid algorithm (HA) integrated genetic algorithm (GA) with ant colony optimization (ACO) for solving Traveling Salesman Problems(TSP) was studied to get better optimization performance than each single algorithm, and complement merits each other and avoid each own demerits. The hybrid algorithm runs GA first and then ACO. A new strategy called GSA was proposed aiming at the key link in the HA that converts genetic solution from GA into information pheromone to distribute in ACO. GSA takes new matrix which is formed by the combination of the former 90% of individual from genetic solution and 10% of individual by random generation as the basis of transformation of pheromone value. The best combination of genetic operators in GA was also discussed. Several TSP were used as simulation tests to test genetic operators matching and optimization performance of HA. The results show that PMX crossover matched with IVM mutation in the GA is the best combination of genetic operators which is able to make GA improve the precision of optimal solution, and HA using the best combination operators and GSA strategy is successful and available to search for optimal solution in high efficiency and has good convergence.