Decision-making of online rescheduling procedures using neuroevolution of augmenting topologies

The use of numerical prediction models are essential to modern society. Data assimilation is a technique that aims to increase the prediction accuracy by combining a model output with observational data, resulting in a state that is closer to the true state of the problem. Depending on the size of the model output and the number of observations to assimilate, the combination of these two sources of information may require intensive computing and become a challenge, even for supercomputers used in this type of application. Thus neural networks have been proposed as an alternative to perform high quality data assimilation at lower computational cost. This paper investigates the use of NeuroEvolution of Augmenting Topologies (NEAT) in data assimilation. NEAT is capable of adapting the connections weights and the neural network topology using principles of evolutionary computation in a search for a minimum topology and best performance. In this work, two different models were used for testing: the Lorenz Attractor and Shallow Water model. The experiments compared the results obtained with NEAT and backpropagation neural networks, using as benchmark the Best Linear Unbiased Estimator (BLUE). In the experiment with the Lorenz Attractor, NEAT was able to emulate the data assimilation task with smaller error at lower computational cost. For the Shallow Water model, tested using different grid sizes, it was observed that the errors obtained with both neural networks were small, but NEAT showed high error values. On the other hand, NEAT always gets a topology with significantly fewer operations, and the computational cost difference increases with the grid size.

Nowadays artificial intelligence (AI) has been applied to many areas, including digital game with various genres. One of them is the fighting game. Implementation of AI in the fighting game aims to make players who are not constrained by the limiting factors which humans possess, such as speed of reflexes and hand gestures. One platform that can be used to make fighting game AI is FightingICE. This platform is official platform used in Computational Intelligence and Games (CIG) conference. One approach that can be used in the process of building the AI in the fighting game is opponent modeling, a method that utilize machine learning to make model based on the behavior of the opponent. There is one AI that utilize k-Nearest Neighbor (kNN) algorithm called MizunoAI. This AI has outstanding performance, but it suffers from inability to adjust its parameter automatically. To solve that problem, in this paper the algorithm used for modelling is changed to Neuroevolution of Augmenting Topologies (NEAT). NEAT is an algorithm with Artificial Neural Network (ANN) as basic architecture and Genetic Algorithm (GA) as the architecture optimizer. In the implementation, NEAT will receive six inputs of distance in the x-axis and y-axis, both players' hitpoints, and both players' stamina. Output of NEAT is the probability of all movements that can be done by the opponent. The entire output will be processed by a simulator to determine the best countermeasure based on the predictions. For testing, the AI made will be faced against four AI winners of CIG 2013, namely MizunoAI, Kaiju, T, and SejongAI. The results showed that the AI is capable to do well in modeling process. Although the performance is still behind the MizunoAI, the learning process can be seen along the match.

Neuroevolution is a powerful and general technique for evolving the structure and weights of artificial neural networks. Though neuroevolutionary approaches such as NeuroEvolution of Augmenting Topologies (NEAT) have been successfully applied to various problems including classification, regression, and reinforcement learning problems, little work has explored application of these techniques to larger-scale multiclass classification problems. In this paper, NEAT is evaluated in several multiclass classification problems, and then extended via two ensemble approaches: One-vs-All and One-vs-One. These approaches decompose multiclass classification problems into a set of binary classification problems, in which each binary problem is solved by an instance of NEAT. These ensemble models exhibit reduced variance and increasingly superior accuracy as the number of classes increases. Additionally, higher accuracy is achieved early in training, even when artificially constrained for the sake of fair comparison with standard NEAT. However, because the approach can be trivially distributed, it can be applied quickly at large scale to solve real problems. In fact, these approaches are incorporated into DarwinTM, an enterprise automatic machine learning solution that also incorporates various other algorithmic enhancements to NEAT. The resulting complete system has proven robust to a wide variety of client datasets.

We used multiobjective genetic algorithms with neuroevolution of augmenting topologies (NEAT) to evolve effective micro behaviors for opposing groups with heterogeneous compositions in StarCraft II, an RTS game. We used the Fast Nondominated Sorting Genetic Algorithm to maximize damage done and minimize damage received in a skirmish, and used this two objective fitness to guide NEAT to evolve the structure and weights of a neural network based controller. The evolved NEAT network controls the movement and attack commands for each unit. We show that non-dominated selection and NEAT can be used together to generate effective micro for groups with two types or three types of units on each side. The evolved micro also generalized well to random configurations, doing well along both objectives. We also manually co-evolved against the best performing individuals produced during a run for multiple cycles and show that this improves micro resulting in better performance against the default Starcraft II AI.

NeuroEvolution of augmenting topologies for solving a two-stage hybrid flow shop scheduling problem: A comparison of different solution strategies

The goal of this paper is to create a hybrid system based on a Multi-Agent Architecture that will investigate the evolution of some neural network methods along with technical and fundamental analysis methods on stock market indexes and how this information influences the stock market behavior in order to improve the profitability of a short or medium time period investment. The proposed system compares the results of Standard Feed Forward Neural Network, Elman and Jordan Recurrent Neural Networks and a Neural Network evolved with Neuro Evolution of Augmenting Topologies (NEAT) in order to investigate which network gives the most accurate result and time performance by taking in consideration the close price of a stock. The system also finds correlations between the pattern recognition methods and technical and fundamental methods results in order to find the direction of the market trend, to predict the next day price of a stock and to trigger a useful buy/sell signal. We are also interested in finding a correlation between the evolution of price, volume, number of transactions in order to have a better view on which is the effect of stock liquidity on a stock price. In order to validate our model a prototype was developed and applied to the Bucharest Stock Exchange Market indexes.

Heart rate (HR) signal analysis is widely used in the medicine and medical research area. Physical activities (PA) are commonly recognized to greatly affect the changes of heart rate. A method of Evolutionary Neural Network -- Neuroevolution of Augmenting Topologies (NEAT) is used to build a PA-based HR predictor model. Through special coding, crossover and mutation operator, NEAT can implement network topology and connectivity weights evolution simultaneously. The common problem in evolutionary neural network, like competing conventions, how to protect the new innovation are effectively solved. The experimental results demonstrated the application potential of the approach.

Cars are becoming more and more intelligent, embedded with a range of sensors to give them local perception of their environment (LIDARs, cameras, etc.). Trendy companies like Google and Tesla are actively testing cars on American roads that can drive without any human interaction [1]. Neural networks are the modern approach for autonomous cars. However, an inefficient neural network algorithm will make the learning process slower and will result in a less reliable autonomous vehicle. In this paper, we will introduce a platform built in JAVA named LAOP (Learning Algorithm Optimization Platform) [2] while explaining the solutions we found to make it easy for researchers to test and compare their own algorithms. Then, we will show how we have integrated a natural selection algorithm with a neural network in order to improve them. Moreover, we will demonstrate how the Fully Connected Neural Network and the NeuroEvolution of Augmenting Topologies (NEAT) [3] algorithms are implemented in the context of vehicular learning on LAOP. Finally, we will display the different results extracted from LAOP by tuning several various parameters such as the weight mutation chance and the car density in the simulation.

A general method to learn the inverse kinematics of multi-link robots by means of neuro-controllers is presented. We can find analytical solutions for the most used and known robots in the bibliography. However, these solutions are specific to a particular robot configuration and are not generally applicable to other robot morphologies. The proposed method is general in the sense that it is not dependant on the robot morphology. We base our method in the Evolutionary Computation paradigm for obtaining incrementally better neuro-controllers. Furthermore, the proposed method solves some very specific issues in robotic neuro-controller learning. (1) It allows to escape from any neural network learning algorithm which relies on the classical supervised input-target learning scheme and hence it lets to obtain neuro-controllers without providing targets or correct answers which -in this case- are un known in prior. (2) It can converge beyond local optimal solutions which is one of the main drawbacks of some neural-network training algorithms based on gradient descent when applied to highly redundant robot morphologies. (3) Using learning algorithms such as the Neuro-Evolution of Augmenting Topologies (NEAT) it is also possible learning the neural network topology on-the-fly which is a common source of empirical testing in neuro-controllers design. Finally, experimental results are provided by applying the method in two multi-link robot learning tasks with a comparison between fixed and learnable topologies.

The characteristics of soft robots make them better candidates for applications such as healthcare, due to their enhanced safety, adaptability, and more natural human‐robot interaction compared to traditional counterparts. Different actuating systems have been proposed for soft robotics. On the other hand, since this technology is fairly young, the design process of soft actuators is not yet well formalized. In an attempt to enhance the applicability of this type of actuator, the utilization of a NeuroEvolution algorithm to automatically design them is proposed here. More specifically, Hypercube‐based NeuroEvolution of Augmented Topologies (HyperNEAT) is investigated for different substrate architectures. These substrates are Artificial Neural Networks that encode the three‐dimensional representation of the soft actuators. The produced three‐dimensional sketches are tested within a simulated environment under two different targets (the maximum displacement and the combination of maximum displacement and minimum actuator volume) to identify the suitability of HyperNEAT as an efficient designing methodology. Since the evaluation of candidate solutions under a physics simulator is the most computationally demanding process, the proposed methodology was realized under a client‐server setting, with the aim of accelerating the evolutionary optimization of actuator sketches. The evaluation part of the algorithm was outsourced to the server side, which can be a specialized and high‐performing computational entity. The resulting soft actuators of this study proved to be of higher competence when compared with actuators derived under previously published evolutionary methodologies.

Current malware detection software often relies on machine learning, which is seen as an improvement over signature-based techniques. Problems with a machine learning based approach can arise when malware writers modify their code with the intent to evade detection. This leads to a cat and mouse situation where new models must constantly be trained to detect new malware variants. In this research, we experiment with genetic algorithms as a means of evolving machine learning models to detect malware. Genetic algorithms, which simulate natural selection, provide a way for models to adapt to continuous changes in a malware families, and thereby improve detection rates. Specifically, we use the Neuro-Evolution of Augmenting Topologies (NEAT) algorithm to optimize machine learning classifiers based on decision trees and neural networks. We compare the performance of our NEAT approach to standard models, including random forest and support vector machines.

Opponent models are necessary in games where the game state is only partially known to the player, since the player must infer the state of the game based on the opponents actions. This paper presents an architecture and a process for developing neural network game players that utilize explicit opponent models in order to improve game play against unseen opponents. The model is constructed as a mixture over a set of cardinal opponents, i.e. opponents that represent maximally distinct game strategies. The model is trained to estimate the likelihood that the opponent will make the same move as each of the cardinal opponents would in a given game situation. Experiments were performed in the game of Guess It, a simple game of imperfect information that has no optimal strategy for defeating specific opponents. Opponent modeling is therefore crucial to play this game well. Both opponent modeling and game-playing neural networks were trained using NeuroEvolution of Augmenting Topologies (NEAT). The results demonstrate that game-playing provided with the model outperform networks not provided with the model when played against the same previously unseen opponents. The cardinal mixture architecture therefore constitutes a promising approach for general and dynamic opponent modeling in game-playing.

In this paper, we compare two neuroevolutionary algorithms, namely standard NeuroEvolution (NE) and NeuroEvolution of Augmenting Topologies (NEAT), with three neurocoevolutionary algorithms, namely Symbiotic Adaptive Neuro-Evolution (SANE), Enforced Sub-Populations (ESP) and Evolving Efficient Connections (EEC). EEC is a novel neurocoevolutionary algorithm that we propose in this work, where the connection weights and the connection paths of networks are evolved separately. All these methods are applied to evolve players of two different board games. The results of this study indicate that neurocoevolutionary algorithms outperform neuroevolutionary algorithms for both domains. Our new method, especially, demonstrates that fully connected networks could generate noise which results in inefficient learning. The performance of standard NE model has been improved significantly through evolving connection weights and efficient connection paths in parallel in our method.

Tool construction requires sophisticated cognitive function and is only observed in higher mammals and a few avian species. In this paper, we will examine the spontaneous emergence of tool construction during the simulated evolution of a two-degree-of-freedom articulated limb controller in a reaching task environment. The limb controller is a recurrent neural network with a topology evolved using the NeuroEvolution of Augmenting Topologies (NEAT) algorithm. First, we show how broad fitness criteria such as distance to target, number of successful reaches, number of steps to reach the target, and number of instances holding the correct length tool are enough to give rise to tool construction. Second, we analyze how the number of tools and their location in the environment during evolution affect the evolved neural circuits' ability to detect tool affordances and employ the optimal decision strategy. We expect our results to help us understand the implications of tool use capability and the environmental conditions that may facilitate its development.

Learning algorithms are being increasingly adopted in various applications. However, further expansion will require methods that work more automatically. To enable this level of automation, a more powerful solution representation is needed. However, by increasing the representation complexity, a second problem arises. The search space becomes huge, and therefore, an associated scalable and efficient searching algorithm is also required. To solve both the problems, first a powerful representation is proposed that unifies most of the neural networks features from the literature into one representation. Second, a new diversity preserving method called spectrum diversity is created based on the new concept of chromosome spectrum that creates a spectrum out of the characteristics and frequency of alleles in a chromosome. The combination of spectrum diversity with a unified neuron representation enables the algorithm to either surpass or equal NeuroEvolution of Augmenting Topologies on all of the five classes of problems tested. Ablation tests justify the good results, showing the importance of added new features in the unified neuron representation. Part of the success is attributed to the novelty-focused evolution and good scalability with a chromosome size provided by spectrum diversity. Thus, this paper sheds light on a new representation and diversity preserving mechanism that should impact algorithms and applications to come.