Evolving explicit opponent models in game playing

Nowadays artificial intelligence (AI) has been applied to many areas, including digital game with various genres. One of them is the fighting game. Implementation of AI in the fighting game aims to make players who are not constrained by the limiting factors which humans possess, such as speed of reflexes and hand gestures. One platform that can be used to make fighting game AI is FightingICE. This platform is official platform used in Computational Intelligence and Games (CIG) conference. One approach that can be used in the process of building the AI in the fighting game is opponent modeling, a method that utilize machine learning to make model based on the behavior of the opponent. There is one AI that utilize k-Nearest Neighbor (kNN) algorithm called MizunoAI. This AI has outstanding performance, but it suffers from inability to adjust its parameter automatically. To solve that problem, in this paper the algorithm used for modelling is changed to Neuroevolution of Augmenting Topologies (NEAT). NEAT is an algorithm with Artificial Neural Network (ANN) as basic architecture and Genetic Algorithm (GA) as the architecture optimizer. In the implementation, NEAT will receive six inputs of distance in the x-axis and y-axis, both players' hitpoints, and both players' stamina. Output of NEAT is the probability of all movements that can be done by the opponent. The entire output will be processed by a simulator to determine the best countermeasure based on the predictions. For testing, the AI made will be faced against four AI winners of CIG 2013, namely MizunoAI, Kaiju, T, and SejongAI. The results showed that the AI is capable to do well in modeling process. Although the performance is still behind the MizunoAI, the learning process can be seen along the match.

The use of numerical prediction models are essential to modern society. Data assimilation is a technique that aims to increase the prediction accuracy by combining a model output with observational data, resulting in a state that is closer to the true state of the problem. Depending on the size of the model output and the number of observations to assimilate, the combination of these two sources of information may require intensive computing and become a challenge, even for supercomputers used in this type of application. Thus neural networks have been proposed as an alternative to perform high quality data assimilation at lower computational cost. This paper investigates the use of NeuroEvolution of Augmenting Topologies (NEAT) in data assimilation. NEAT is capable of adapting the connections weights and the neural network topology using principles of evolutionary computation in a search for a minimum topology and best performance. In this work, two different models were used for testing: the Lorenz Attractor and Shallow Water model. The experiments compared the results obtained with NEAT and backpropagation neural networks, using as benchmark the Best Linear Unbiased Estimator (BLUE). In the experiment with the Lorenz Attractor, NEAT was able to emulate the data assimilation task with smaller error at lower computational cost. For the Shallow Water model, tested using different grid sizes, it was observed that the errors obtained with both neural networks were small, but NEAT showed high error values. On the other hand, NEAT always gets a topology with significantly fewer operations, and the computational cost difference increases with the grid size.

Neuroevolution is a powerful and general technique for evolving the structure and weights of artificial neural networks. Though neuroevolutionary approaches such as NeuroEvolution of Augmenting Topologies (NEAT) have been successfully applied to various problems including classification, regression, and reinforcement learning problems, little work has explored application of these techniques to larger-scale multiclass classification problems. In this paper, NEAT is evaluated in several multiclass classification problems, and then extended via two ensemble approaches: One-vs-All and One-vs-One. These approaches decompose multiclass classification problems into a set of binary classification problems, in which each binary problem is solved by an instance of NEAT. These ensemble models exhibit reduced variance and increasingly superior accuracy as the number of classes increases. Additionally, higher accuracy is achieved early in training, even when artificially constrained for the sake of fair comparison with standard NEAT. However, because the approach can be trivially distributed, it can be applied quickly at large scale to solve real problems. In fact, these approaches are incorporated into DarwinTM, an enterprise automatic machine learning solution that also incorporates various other algorithmic enhancements to NEAT. The resulting complete system has proven robust to a wide variety of client datasets.

Decision-making of online rescheduling procedures using neuroevolution of augmenting topologies

We used multiobjective genetic algorithms with neuroevolution of augmenting topologies (NEAT) to evolve effective micro behaviors for opposing groups with heterogeneous compositions in StarCraft II, an RTS game. We used the Fast Nondominated Sorting Genetic Algorithm to maximize damage done and minimize damage received in a skirmish, and used this two objective fitness to guide NEAT to evolve the structure and weights of a neural network based controller. The evolved NEAT network controls the movement and attack commands for each unit. We show that non-dominated selection and NEAT can be used together to generate effective micro for groups with two types or three types of units on each side. The evolved micro also generalized well to random configurations, doing well along both objectives. We also manually co-evolved against the best performing individuals produced during a run for multiple cycles and show that this improves micro resulting in better performance against the default Starcraft II AI.

NeuroEvolution of augmenting topologies for solving a two-stage hybrid flow shop scheduling problem: A comparison of different solution strategies

The goal of this paper is to create a hybrid system based on a Multi-Agent Architecture that will investigate the evolution of some neural network methods along with technical and fundamental analysis methods on stock market indexes and how this information influences the stock market behavior in order to improve the profitability of a short or medium time period investment. The proposed system compares the results of Standard Feed Forward Neural Network, Elman and Jordan Recurrent Neural Networks and a Neural Network evolved with Neuro Evolution of Augmenting Topologies (NEAT) in order to investigate which network gives the most accurate result and time performance by taking in consideration the close price of a stock. The system also finds correlations between the pattern recognition methods and technical and fundamental methods results in order to find the direction of the market trend, to predict the next day price of a stock and to trigger a useful buy/sell signal. We are also interested in finding a correlation between the evolution of price, volume, number of transactions in order to have a better view on which is the effect of stock liquidity on a stock price. In order to validate our model a prototype was developed and applied to the Bucharest Stock Exchange Market indexes.

Heart rate (HR) signal analysis is widely used in the medicine and medical research area. Physical activities (PA) are commonly recognized to greatly affect the changes of heart rate. A method of Evolutionary Neural Network -- Neuroevolution of Augmenting Topologies (NEAT) is used to build a PA-based HR predictor model. Through special coding, crossover and mutation operator, NEAT can implement network topology and connectivity weights evolution simultaneously. The common problem in evolutionary neural network, like competing conventions, how to protect the new innovation are effectively solved. The experimental results demonstrated the application potential of the approach.

Cars are becoming more and more intelligent, embedded with a range of sensors to give them local perception of their environment (LIDARs, cameras, etc.). Trendy companies like Google and Tesla are actively testing cars on American roads that can drive without any human interaction [1]. Neural networks are the modern approach for autonomous cars. However, an inefficient neural network algorithm will make the learning process slower and will result in a less reliable autonomous vehicle. In this paper, we will introduce a platform built in JAVA named LAOP (Learning Algorithm Optimization Platform) [2] while explaining the solutions we found to make it easy for researchers to test and compare their own algorithms. Then, we will show how we have integrated a natural selection algorithm with a neural network in order to improve them. Moreover, we will demonstrate how the Fully Connected Neural Network and the NeuroEvolution of Augmenting Topologies (NEAT) [3] algorithms are implemented in the context of vehicular learning on LAOP. Finally, we will display the different results extracted from LAOP by tuning several various parameters such as the weight mutation chance and the car density in the simulation.

A general method to learn the inverse kinematics of multi-link robots by means of neuro-controllers is presented. We can find analytical solutions for the most used and known robots in the bibliography. However, these solutions are specific to a particular robot configuration and are not generally applicable to other robot morphologies. The proposed method is general in the sense that it is not dependant on the robot morphology. We base our method in the Evolutionary Computation paradigm for obtaining incrementally better neuro-controllers. Furthermore, the proposed method solves some very specific issues in robotic neuro-controller learning. (1) It allows to escape from any neural network learning algorithm which relies on the classical supervised input-target learning scheme and hence it lets to obtain neuro-controllers without providing targets or correct answers which -in this case- are un known in prior. (2) It can converge beyond local optimal solutions which is one of the main drawbacks of some neural-network training algorithms based on gradient descent when applied to highly redundant robot morphologies. (3) Using learning algorithms such as the Neuro-Evolution of Augmenting Topologies (NEAT) it is also possible learning the neural network topology on-the-fly which is a common source of empirical testing in neuro-controllers design. Finally, experimental results are provided by applying the method in two multi-link robot learning tasks with a comparison between fixed and learnable topologies.

Current malware detection software often relies on machine learning, which is seen as an improvement over signature-based techniques. Problems with a machine learning based approach can arise when malware writers modify their code with the intent to evade detection. This leads to a cat and mouse situation where new models must constantly be trained to detect new malware variants. In this research, we experiment with genetic algorithms as a means of evolving machine learning models to detect malware. Genetic algorithms, which simulate natural selection, provide a way for models to adapt to continuous changes in a malware families, and thereby improve detection rates. Specifically, we use the Neuro-Evolution of Augmenting Topologies (NEAT) algorithm to optimize machine learning classifiers based on decision trees and neural networks. We compare the performance of our NEAT approach to standard models, including random forest and support vector machines.

In this paper, we compare two neuroevolutionary algorithms, namely standard NeuroEvolution (NE) and NeuroEvolution of Augmenting Topologies (NEAT), with three neurocoevolutionary algorithms, namely Symbiotic Adaptive Neuro-Evolution (SANE), Enforced Sub-Populations (ESP) and Evolving Efficient Connections (EEC). EEC is a novel neurocoevolutionary algorithm that we propose in this work, where the connection weights and the connection paths of networks are evolved separately. All these methods are applied to evolve players of two different board games. The results of this study indicate that neurocoevolutionary algorithms outperform neuroevolutionary algorithms for both domains. Our new method, especially, demonstrates that fully connected networks could generate noise which results in inefficient learning. The performance of standard NE model has been improved significantly through evolving connection weights and efficient connection paths in parallel in our method.

Recently, neural network dynamic modeling has drawn a great deal of attention not only from academia but also from industry. It has been shown that neural network can learn to mimic the behavior of a complex chemical process. However, a neural network dynamic model might be difficult to develop when training data do not contain sufficient nonlinear information about the process. In addition, owing to the capacity of a modern control system, a great deal of steady-state data is stored. Most modeling techniques to date do not take into account the fact that the transient data may not contain sufficient nonlinear information and that the nonlinear steady-state data are abundant and easily accessible. In order to fully utilize the abundant steady-state information, we present a neural network Hammerstein (NNH) modeling approach. In the proposed approach, a static neural network is integrated with a linear dynamic model in series. A constrained gradient-based algorithm is derived for training the linear dynamic model in order to retain the nonlinearity established by the static neural network. To demonstrate the proposed modeling approach, a complex polymerization process is used as an example

It may be a good use for setting and changing components of game levels to analyze a FPS game player's goal-oriented action pattern. This article chooses test groups for novice and expert depending on game players' experience and verifies game actions seen through repetitive experiences. Furthermore, the actions which appear group by group are classified and coded so that the level of interaction with players according to game level may be ascertained with their change process. Accordingly, this study may be helpful to suggest a detailed modification method for correct expectation and game level in the situation of each game play.

Building engineers and managers often treat occupants as passive agents despite their important role in building energy performance. Meanwhile, neglecting occupants' preferences may lead to poor implementations of building controls. The overall goal of this research is to improve buildings' energy performance and occupants' satisfaction by considering occupant-building interactions. This research first assessed the impact of static and dynamic occupant modelling approaches in a simulation-based analysis using the existing occupant models. The results showed the discrepancies between these two approaches in predicting the energy performance. The analysis emphasized the importance of using dynamic modelling approach. Given that the existing occupant models are context-specific, the next steps of this research focused on conducting a monitoring campaign. Therefore, the various monitoring methods exist in the literature and the anecdotal evidence were critically reviewed. Among the methods, environmental conditions are more controllable as well as configuring building designs and control systems are more flexible with laboratories and virtual environments. However, researchers can achieve valuable information on the natural occupants' presence and behaviour by in-situ monitoring on a relatively large sample size at a lower cost in long-term studies. This research adopted the in-situ method in a case study.