Agents for Games and Simulations

Game Agent is currently being developed to be an opponent in the game, including games with the Card Games genre. Game Agents on traditional Card Games - such as poker, dominoes, or mahjong cards - have abilities that depend on the value of the cards, but the ability of these Game Agents will not be optimal if used in the Card Battle game. This is because Card Battle has many attributes that must be processed to become opponents. Therefore, this research modifies the Game Agent with Genetic Algorithm to optimize the playing ability of the Game Agent in Card Battle. The computational stages and fitness formula of the Genetic Algorithm are adjusted to the Card Battle rules to increase the computational speed of the Genetic Algorithm. The results of this study prove that Game Agent modification of Genetic Algorithm provides a more optimal playing ability than its predecessor algorithm. Game Agent that has been modified has several abilities that are not owned by the previous Game Agent, such as issuing cards to attack opponents directly and storing SP (Summon Points) they have.

This paper firstly seeks an answer to the question of whether Non Player Characters (NPCs) in Computer Games can also be viewed as game agents where reactivity, autonomy, being temporarily continuous and having goal-oriented behavior are taken as the features of being a game agent. Within this scope, we will try to assess whether naming NPCs as agents point to a desire that one day they would fulfill the requirements of being an agent or whether NPCs actually already fulfill these requirements. Secondly, the paper looks into the AI needs of video games. We present the AI methodologies that are either being used and under research for use in Game AI. The same methodologies are also likely to contribute to the increasing levels of autonomy and goal-directed behavior of NPCs and help them become more agent-like.

In this paper, we present an approach of adaptive learning mechanism for game agents' real-time behavior control. This approach mainly focuses on how to generate game agent's adaptability in real-time. It is possible to apply our approach in complicated game character interactions by following the framework discussed in this paper. We consider the layered architecture, the behavior pattern and the adaptive mechanism design to be the three key points of our approach. We provide a brief example of how to apply adaptive learning in game agents' behavior processing. From this example, we demonstrate that the planning and learning process is fast enough to have 3D model rendered in time.

Combined prosocial incentives, integrating reward for cooperators and punishment for defectors, are effective tools to promote cooperation among competing agents in population games. Existing research concentrated on how to adjust reward or punishment, as two mutually exclusive tools, during the evolutionary process to achieve the desired proportion of cooperators in the population, and less attention has been given to exploring a combined incentive-based control policy that can steer the system to the full cooperation state at the lowest cost. In this work, we propose a combined incentive scheme in a population of agents whose conflicting interactions are described by the prisoner's dilemma game on complete graphs and regular networks, respectively. By devising an index function for quantifying the implementation costs of the combined incentives, we analytically construct the optimally combined incentive protocol by using optimal control theory. By means of theoretical analysis, we identify the mathematical conditions, under which the optimally combined incentive scheme requires the minimal amount of cost. In addition to numerical calculations, we further perform computer simulations to verify our theoretical results and explore their robustness on different types of network structures.

Agents for Games and Simulations II

In realistic social game environments, agents need to exhibit a certain level of social awareness to maintain the illusion of life and to provide important interaction cues. Previously we have reported on an engine that imbues game agents with socially reactive behavior based on a continuous steering framework and a model of human territoriality. However, some social action requires deliberation and planning. In this paper we describe an addition to our engine that provides dynamic social planning on top of the reactive layer. We propose a novel approach that considers both the emotional and social impact of events and actions on the agent. An implementation demonstrates the approach.

As a step toward simulating dynamic dialogue between agents and humans in virtual environments, we describe learning a model of social behavior composed of interleaved utterances and physical actions. In our model, utterances are abstracted as {speech act, propositional content, referent} triples. After training a classifier on 100 gameplay logs from The Restaurant Game annotated with dialogue act triples, we have automatically classified utterances in an additional 5,000 logs. A quantitative evaluation of statistical models learned from the gameplay logs demonstrates that semiautomatically classified dialogue acts yield significantly more predictive power than automatically clustered utterances, and serve as a better common currency for modeling interleaved actions and utterances.

Computer games are becoming increasingly popular as a research platform for agents research and applications. A key problem for game agents is responding in a timely and appropriate way to multiple, often conflicting goals in a complex, dynamic environment. In this paper we propose a novel goal processing architecture for game agents. Building on the teleo-reactive programming framework originally developed in robotics, we introduce the notion of a resource, which can be used to gain exclusive access to specific game objects as well as to represent more abstract things such as properties of the agent. We then describe a goal arbitration architecture for teleo-reactive programs with resources. Our architecture allows an agent to respond flexibly to multiple competing goals, and simplifies the development of game agents by facilitating increased code re-use.

Game agents are efficiently implemented through different AI techniques, such as neural network, reinforcement learning, and evolutionary search. Although there are many works for each approach, we present a critical analysis and comparison between them, suggesting a common benchmark and parameter configurations. The evolutionary strategy implements the NeuroEvolution of Augmenting Topologies algorithm, while the reinforcement learning agent leverages Q-Learning and Proximal Policy Optimization. We formulate and empirically compare this set of solutions using the Flappy Bird game as a test scenario. We also compare different representations of state and reward functions for each method. All methods were able to generate agents that can play the game, where the NEAT algorithm had the best results, reaching the goal of never losing.

This paper proposes a new application of the stochastic game model to solve the problem of self- organization of the Hamiltonian cycle of a graph. To do this, at the vertices of the undirected graph are placed game agents, whose pure strategies are options for choosing one of the incident edges. A random selection of strategies by all agents forms a set of local paths that begin at each vertex of the graph. Current player payments are defined as loss functions that depend on the strategies of neighboring players that control adjacent vertices of the graph. These functions are formed from a penalty for the choice of opposing strategies by neighboring players and a penalty for strategies that have reduced the length of the local path. Random selection of players’ pure strategies is aimed at minimizing their average loss functions. The generation of sequences of pure strategies is performed by a discrete distribution built on the basis of dynamic vectors of mixed strategies. The elements of the vectors of mixed strategies are the probabilities of choosing the appropriate pure strategies that adaptively take into account the values of current losses. The formation of vectors of mixed strategies is determined by the Markov recurrent method, for the construction of which the gradient method of stochastic approximation is used. During the game, the method increases the value of the probabilities of choosing those pure strategies that lead to a decrease in the functions of average losses. For given methods of forming current payments, the result of the stochastic game is the formation of patterns of self-organization in the form of cyclically oriented strategies of game agents. The conditions of convergence of the recurrent method to collectively optimal solutions are ensured by observance of the fundamental conditions of stochastic approximation. The game task is extended to random graphs. To do this, the vertices are assigned the probabilities of recovery failures, which cause a change in the structure of the graph at each step of the game. Realizations of a random graph are adaptively taken into account when searching for Hamiltonian cycles. Increasing the probability of failure slows down the convergence of the stochastic game. Computer simulation of the stochastic game provided patterns of self-organization of agents’ strategies in the form of several local cycles or a global Hamiltonian cycle of the graph, depending on the ways of forming the current losses of players. The reliability of experimental studies is confirmed by the repetition of implementations of self-organization patterns for different sequences of random variables. The results of the study can be used in practice for game-solving NP-complex problems, transport and communication problems, for building authentication protocols in distributed information systems, for collective decision-making in conditions of uncertainty.

This paper explores the possibilities for believable game agents (NPCs) through the implementation of a Model Social Game Agent (MSGA). We present a high level model focusing on the conceptual framework for implementing MSGAs on a Second Life server.

A transfer learning approach is presented to address the challenge of training video game agents with limited data. The approach decomposes games into objects, learns object models, and transfers models from known games to unfamiliar games to guide learning. Experiments show that the approach improves prediction accuracy over a comparable control, leading to more efficient exploration. Training of game agents is thus accelerated by transferring object models from previously learned games.

In this paper, we consider the following problem - what affects the Nash equilibrium amount of investment in knowledge when one of the complete graph enters another full one. The solution of this problem will allow us to understand exactly how game agents will behave when deciding whether to enter the other net, what conditions and externalities affect it and how the level of future equilibrium amount of investments in knowledge can be predicted.

Advancements in artificial intelligence are leading researchers to find use cases that were not as straightforward to solve in the past. The use case of simulated autonomous driving has been known as a notoriously difficult task to automate, but advancements in the field of reinforcement learning have made it possible to reach satisfactory results. In this paper, we explore the use of the Unity ML-Agents toolkit to train intelligent agents to navigate a racing track in a simulated environment using RL algorithms. The paper compares the performance of several different RL algorithms and configurations on the task of training kart agents to successfully traverse a racing track and identifies the most effective approach for training kart agents to navigate a racing track and avoid obstacles in that track. The best results, value loss of 0.0013 and a cumulative reward of 0.761, were yielded using the Proximal Policy Optimization algorithm. After successfully choosing a model and algorithm that can traverse the track with ease, different objects were added to the track and another model (which used behavioral cloning as a pre-training option) was trained to avoid such obstacles. The aforementioned model resulted in a value loss of 0.001 and a cumulative reward of 0.068, proving that behavioral cloning can help achieve satisfactory results where the in game agents are able to avoid obstacles more efficiently and complete the track with human-like performance, allowing for a deployment of intelligent agents in racing simulators.

The research titled "Bajau-tainment" Educational Game (Edu-game) is a research project that focuses on the development of a Puzzle Game designed to improve memory, specifically in language learning. In this game, players must arrange randomized letters to form a word in the Bajau language. This research applies the shuffle random algorithm, which aims to ensure that the arrangement of letters is always shuffled, making the gameplay dynamic, non-monotonous, and engaging. AI Artificial Intelligence (AI) technology will also be applied in this research. Using the Finite State Machine (FSM) model method, the game will feature a game agent character that will accompany children during gameplay, like how a friend accompanies students in a classroom. This virtual friend can display emotions such as happiness or sadness based on the game environment. The research aims to make this edu-game more appealing and interactive for children. The AI game agent will act as a friend who accompanies the child throughout the game.