Smart Terrain in Online Tactic Agent decision making process

For the last few decades, there was tremendous interest in harnessing 3D virtual environments for education and training. Multi-user game-like environments make use of non-player characters and quest activities in tutoring and training. This approach incorporates successful active learning and incremental progress. This article presents issues encountered while adapting the multi-user online game genre for educational virtual environments. In this endeavour, non-player characters play a central role in organising and delivering educational content. Educational virtual environments call for additional kinds of extension beyond generic user content creation as delivered in some virtual worlds like Second Life. Support for these environments will emphasise the creation of new activities. A set of libraries and techniques named portable non-player character tutors and quests reduces the effort needed to develop and integrate educational non-player characters. These non-player characters are introduced in CVE, a platform for rapidly developing educational virtual environments. The portable non-player character tutors and quests framework enables formatted web-based exercises, quizzes, and educational content to be incorporated into virtual worlds, reducing the effort needed to create new content. The framework presented in this article introduces initial non-player characters that support educational quest activities to the 3D environments users.

In current state-of-the-art commercial first person shooter games, computer controlled bots, also known as nonplayer characters, can often be easily distinguishable from those controlled by humans. Tell-tale signs such as failed navigation, “sixth sense” knowledge of human players' whereabouts and deterministic, scripted behaviors are some of the causes of this. We propose, however, that one of the biggest indicators of nonhumanlike behavior in these games can be found in the weapon shooting capability of the bot. Consistently perfect accuracy and “locking on” to opponents in their visual field from any distance are indicative capabilities of bots that are not found in human players. Traditionally, the bot is handicapped in some way with either a timed reaction delay or a random perturbation to its aim, which doesn't adapt or improve its technique over time. We hypothesize that enabling the bot to learn the skill of shooting through trial and error, in the same way a human player learns, will lead to greater variation in game-play and produce less predictable nonplayer characters. This paper describes a reinforcement learning shooting mechanism for adapting shooting over time based on a dynamic reward signal from the amount of damage caused to opponents.

Developing a virtual 3D environment by using game engine is a strategy to incorporate various multimedia data into one platform. The characteristic of game engine that is preinstalled with interactive and navigation tools allows users to explore and engage with the game objects. However, most CAD and GIS applications are not equipped with 3D tools and navigation systems intended to the user experience. In particular, 3D game engines provide standard 3D navigation tools as well as any programmable view to create engaging navigation thorough the virtual environment. By using a game engine, it is possible to create other interaction such as object manipulation, non playing character (NPC) interaction with player and/or environment. We conducted analysis on previous game engines and experiment on urban design project with Unity3D game engine for visualization and interactivity. At the end, we present the advantages and limitations using game technology as visual representation tool for architecture and urban design studies.

In this paper, we introduce a skill-balancing mechanism for adversarial non-player characters (NPCs), called Skilled Experience Catalogue (SEC). The objective of this mechanism is to approximately match the skill level of an NPC to an opponent in real-time. We test the technique in the context of a First-Person Shooter (FPS) game. Specifically, the technique adjusts a reinforcement learning NPC's proficiency with a weapon based on its current performance against an opponent. Firstly, a catalogue of experience, in the form of stored learning policies, is built up by playing a series of training games. Once the NPC has been sufficiently trained, the catalogue acts as a timeline of experience with incremental knowledge milestones in the form of stored learning policies. If the NPC is performing poorly, it can jump to a later stage in the learning timeline to be equipped with more informed decision-making. Likewise, if it is performing significantly better than the opponent, it will jump to an earlier stage. The NPC continues to learn in real-time using reinforcement learning but its policy is adjusted, as required, by loading the most suitable milestones for the current circumstances.

CHI Systems, under contract to the U. S. Army Research Institute, developed an immersive training system, called Virtual Environment Composable Training for Operational Readiness Training Delivery (VECTOR-TD), which provides scenario-based virtual environments for cultural familiarization. VECTOR-TD was designed to provide a new technology for game-based training in cultural familiarization through the application of scenario-based training. VECTOR-TD integrates the Lithtech Jupiter game engine for the virtual environment and CHI System’s iGEN® cognitive agent development toolkit to implement intelligent game characters and integrated performance monitoring. In order for VECTOR-TD to have true long term value to the Army, it was determined that it would be necessary to develop content management tools which would allow Army training personnel to edit and author VECTOR-TD scenarios. To address this, VECTOR-Scenario Editor (VECTOR-SE) was developed based on VECTOR-TD XML-based scripting language to externalize the representation of the training scenario, the behavior of non-player characters (NPCs) within the scenario, and the behavior of the instructor NPC responsible for performance evaluation and After Action Review (AAR). The XML-based language provides direct control of the behavior, dialog, emotional state, and predispositions of NPCs within the scenario. VECTOR-SE was successfully developed to provide content development and management tools that allow non-programmers to author scenarios and manipulate training parameters. VECTOR-SE is based on an instructional design process model and involves the specification of learning objectives, the creation of scenario segments (i.e., vignettes), the authoring of character/trainee dialog, the designation of dialog branching, the placement of characters at game locations, and the designation of when training assistance/remediation will be delivered. The significance of VECTOR-SE is twofold. First, it drastically reduces the time and skill required to develop VECTOR scenarios. Within the original VECTOR project, a 15-minute cultural training scenario was developed which required 2 months of programming and cognitive modeling effort. The same scenario was later implemented in VECTOR-SE in under 3 days. Additional scenarios with comparable complexity have also been implemented using VECTOR-SE and similar significant reductions in development time have been observed. Second, VECTOR-SE makes scenario development or modification accessible to a wider audience of professionals. VECTOR-TD and SE are currently being evaluated at the U.S. Military Academy at West Point.

Current computer games are being set in increasingly more complex and dynamic virtual environments. Massively multiplayer online games, for example, are played in persistent virtual worlds, which evolve and change as players create and personalize their own virtual property. In contrast, technologies for controlling the behavior of nonplayer characters that populate virtual game worlds are frequently limited to preprogrammed rules. Characters using fixed rule-sets lack the ability to adapt in time with their environment. Motivated reinforcement learning offers an alternative to character design that can achieve nonplayer characters that both evolve and adapt in dynamic environments. This article presents and evaluates two computational models of motivation for use in nonplayer characters in persistent computer game worlds. These models represent motivation as an ongoing search for novelty, interest, and competence. Two metrics are introduced to evaluate the adaptability of characters controlled by motivated reinforcement learning agents using different models of motivation. These metrics characterize the behavior of nonplayer characters in terms of the variety and complexity of learned behaviors. An empirical evaluation of characters in simulated game scenarios shows that characters motivated by the search for competence are more adaptable in dynamic environments than those motivated by interest and novelty alone.

Fire accidents are one of the most frequently occurring disasters leading to life and property loss. Training for fire escape skills and designing efficient building evacuation plans are increasingly important research topics. Virtual environments (VE) are widely used to support training and simulating human response under emergency situations. However, current VE based training systems have limited intelligent non-player characters (NPC) which are lacking in support of fire science knowledge. This project will introduce a hybrid method combining gaming technology, agent programming and fire science knowledge to design an evacuation training system for fire wardens. The success of using professional numerical fire simulation tool to support NPC behaviour in virtual environment will provide a new way to enhance the realism of virtual environment by using high fidelity fire data sources.

We developed a virtual environment (VE) and artificial intelligence (AI) techniques that enables the study of virtual reality (VR) training applications to develop a sense of empathy and empathetic curiosity in nursing students towards patients living with schizophrenia. The VE is a simulated city-street setting providing simulated auditory hallucinations (hearing voices), and interactive animated non-player characters (NPCs). The VE is implemented in the Unity game development platform and is experienced using an OculusRift_S head mounted display (HMD). Our new machine learning (ML) algorithms and deep learning (DL) models provide a simulated vision capability, enabling the primary NPC to recognize the user’s facial expressions and react with a variety of facial, body, and verbal animated responses. Our ML and DL techniques are designed to provide effective facial detection and expression classification when the user’s face is occluded by the HMD. The integration of the VE and AI-based techniques provides the opportunity to create more complex, two-way user-to-NPC interactions in the future. We hypothesize that these enriched interactions provide value by simulating encounters between caregivers and patients when the same experiences are not readily available in a real world clinical environment, thus filling a need for improved empathy training. An IRB study to test our hypothesis is planned by the College of Nursing faculty.

This paper firstly seeks an answer to the question of whether Non Player Characters (NPCs) in Computer Games can also be viewed as game agents where reactivity, autonomy, being temporarily continuous and having goal-oriented behavior are taken as the features of being a game agent. Within this scope, we will try to assess whether naming NPCs as agents point to a desire that one day they would fulfill the requirements of being an agent or whether NPCs actually already fulfill these requirements. Secondly, the paper looks into the AI needs of video games. We present the AI methodologies that are either being used and under research for use in Game AI. The same methodologies are also likely to contribute to the increasing levels of autonomy and goal-directed behavior of NPCs and help them become more agent-like.

Anagram is a turn-based role-playing game where two players construct words by arranging given letters. A significant aspect of playing a game is the challenge. A good challenge comes from an opponent with a close ability. In a two-player game like Anagram, the second player can be a nonhuman player called Non-Playable Character (NPC). A balanced game is more engaging. Therefore, it is imperative to insert artificial intelligence (AI) into an NPC to make it possess a balance ability. This study investigates the AI algorithm that is the most appropriate to make a balance NPC for Anagram games. We tested three scenarios: Descending AI, Random AI, and AI with k-Nearest Neighbour (k-NN). Descending AI gets an Anagram solution by selecting a word with the highest score from all possible answers. Random AI picks a word randomly from the possible answers, while AI with k-NN chooses a word closest to one of the human players. The results show that Descending AI is the best algorithm to make the strongest NPC, which always gets the highest score, followed by Random AI and AI with k-NN. However, AI with the k-NN algorithm makes the constructed NPC has the highest number of turns at an average of 18, while Descending AI gets 14 turns and Random AI has 15 turns. Looking at the remaining lives at the end of the game, AI with k-NN makes the NPC has 25 lives left, while Descending AI has 59 lives, and Random AI has 48 lives. Less remaining lives suggest that NPC containing AI with the k-NN algorithm matches closer to the human player and therefore is more suitable for Anagram NPC.

Non-Playable Characters (NPCs) are a subtype of virtual agents that populate video games by endorsing social roles in the narrative. To infer NPCs’ roles, players evaluate NPCs’ appearance and behaviors, usually by ascribing human traits to NPCs, such as intelligence, likability and morality. In particular, hostile NPCs in video games are essential to build the games’ inherent challenges. The three experiments reported here investigated the extent to which the perception of hostility in a military shooter game (including both threat of appearance and aggressiveness in behaviors) is influenced by the appearance and the behaviors of NPCs thanks to perceived intelligence, likability and morality-related questionnaires. Our results first show that hostility is efficiently conveyed through NPCs’ behaviors, but not significantly by their appearance. Second, our study allows identifying the main predictors of hostility perception, namely unfriendliness, knowledge and harmfulness.

replicating a human-like behavior for a Non-player character (NPC) was always a challenge in sports simulations. Having a non-repetitive and intelligent behavior can dramatically increase the entertainment value of the game. To achieve this, many developers used some rule based or expert system methods which result in a controlled NPC behavior, but recently some machine learning methods are deployed in various types of games. In this paper we propose a method for an NPC to learn main skills in the tennis game. The method is based on the multi layered perceptron (MLP) network. We created a virtual 3D environment of the tennis game which can be interacted with via virtual reality devices. Then, after collecting required data from this environment, implemented a virtual opponent that learns the fundamental skills required to play a tennis game. Finally we compared results of different learning methods.

Non-Player Characters (NPCs) are critical to the interactive and dynamic character of modern video games. They inhabit game worlds, engage with players, and play a large role in storytelling and gameplay mechanics. Yet, conventional rule-based NPCs tend to fall short of realism, flexibility, and scalability needed to keep pace with changing player expectations and sophisticated game worlds. This paper discusses the intersection of Artificial Intelligence (AI) engineering techniques with game development to create intelligent, adaptive NPCs. Through modular AI architectures, reinforcement learning, behaviour cloning, and effective deployment strategies, this research presents a complete framework for developing intelligent NPCs that provide increased interactivity and engagement. It also discusses the application of memory repositories to preserve NPC contextual awareness across extended interactions. Comparative analysis and case studies are employed to illustrate enhancements in gameplay dynamics and player experience.

In this paper, an innovative approach to controlling temperature was made. Using Unity as our chosen platform, we crafted a virtual environment, that replicates the real-world temperature change dynamics. For this, a virtual environment mimics an enclosed area with a virtual heat source. Additionally, this environment’s area and air density were also taken into consideration. Then, a virtual sensor and cooler were added for detecting and adjusting the environment temperature. Finally, the ML agent package was integrated for controlling temperature with RL. This integration consisted of adding a script that inherits from the agent class and has environment, sensor, and cooler as variables. In each epi-sode, there was a time duration in which both environment's temperature change was calculated and the reward was set to be given only when the environment's temperature was at a certain range. The empirical findings show the efficiency of our approach, show-casing the system's ability to adapt and optimize temperature regulation parameters over successive iterations. By successfully integrating Unity and ML agents, our study under-scores the transformative potential of harnessing virtual environments and machine learning for advancing temperature control technologies. Keywords: unity, ml agents, temperature control, virtual environment, reinforcement learning.

First Person Shooter (FPS) games are a popular online gaming genre played predominately over wired networks. FPS games have stringent delay requirements. However, the explosive growth in wireless LAN (WLAN) deployment has seen an increase in the use of such networks for gaming purposes. The varied performance of the IEEE 802.11 standard has tended to make WLANs unsuitable for FPS games. The IEEE 802.11e enhancement introduces Quality of Service (QoS) mechanisms including Enhanced Distributed Channel Access (EDCA) allowing prioritization of competing flows. Using the NS-2 simulator, we evaluate the capability of 802.11 WLANs to support Quake IV games traffic in the presence of web traffic. We compare the results achieved using EDCA with those achieved by a non-elevated differentiated services scheduler known as Best Effort with Loss Trade-off (BELT). We find that the BELT scheduler compares favorably with EDCA in this context.