A Hierarchical Reinforcement Learning Based Artificial Intelligence for Non-Player Characters in Video Games

Creating the behavior for non-player characters in video games is a complex task that requires the collaboration among programmers and game designers.

Authoring the AI for non-player characters (NPCs) in modern video games is an increasingly complex task. Designers and programmers must collaborate to resolve a tension between believable agents with emergent behaviours and scripted story lines. Behaviour trees (BTs) have been proposed as an expressive mechanism that let designers create complex behaviours along the lines of the story they want to tell. However, BTs are still too complex for non-programmers. In this paper, we propose the use of plan traces to assist designers when building BTs. In order to make this approach feasible within state-of-the-art video game technology, we generate the planning domain through an extension of the component-based approach, a widely used technique for representing entities in commercial video games.

Abstract: Applications of fuzzy logic in game creation have proven useful, especially when it comes to modelling the behavior of non-player characters (NPCs) in video games. NPC behavior in video games has historically been governed by strict rule-based systems or scripted sequences, which frequently led to restricted and predictable interactions. On the other hand, fuzzy logic adds a degree of flexibility and ambiguity to the decision-making processes of NPCs. By improving the naturalness, adaptability, and emotional resonance of NPC behavior, fuzzy logic may greatly enhance player experiences in games. It provides players with progressively more engaging and dynamic gaming experiences as it continues to be incorporated into game production, opening up new possibilities for interactive narrative, gaming, and immersive virtual worlds. Based on Utku Kose's article, "Developing a Fuzzy Logic Based Game System," I have investigated the use of fuzzy logic in gaming contexts for NPC decision-making in this study.

Imagine a scenario in which players engage in conversations with non-playable characters (NPCs) in video games using a microphone, free from scripted responses or predefined constraints. With the advancement of Generative Artificial Intelligence (GAI), this possibility has become more feasible than ever. This article examines the Mantella mod for The Elder Scrolls V: Skyrim (Bethesda, 2011), which facilitates natural language interactions with NPCs, transforming player engagement within the game’s fictional universe. Skyrim is an action role-playing game (RPG) developed by Bethesda Game Studios. With over 60 million copies sold and releases spanning three generations of gaming consoles, Skyrim has cemented its place in gaming history. A significant factor contributing to its enduring relevance is its vibrant modding community, which has enabled players to modify and expand the game in myriad ways – ranging from graphical enhancements to the introduction of new areas and survival mechanics. This extensive capacity for customization has sustained the game’s popularity for over a decade since its initial release. The primary objective of this article is to investigate the creative appropriation of GAI by Skyrim's PC modding community. Specifically, it addresses the following question: What AI imaginaries emerge from Skyrim gameplay influenced by the Mantella mod? To explore this question, 13 YouTube videos were collected using ethnographic techniques, complemented by a formal gameplay analysis.

Americans are increasingly concerned about video games, presumably due to the amount and graphicness of violence they contain. Social Cognitive Theory suggests that people are more likely to imitate characters they see as attractive or similar to self. To date, however, little research has examined attributes of violent characters in video games related to this issue. This content analysis examined 10 minutes of play from 60 of the most popular video games. Adapting the coding scheme from the National Television Violence Study (Smith et al., 1998; Wilson et al., 1997, 1998), various demographic and contextual features of violent characters and violent interactions were assessed. The results show that violent game characters have attributes that are likely to increase the extent to which some players perceive them as attractive and similar. Moreover, these violent characters engage in aggression that is presented as justified or graphic.

Anagram is a turn-based role-playing game where two players construct words by arranging given letters. A significant aspect of playing a game is the challenge. A good challenge comes from an opponent with a close ability. In a two-player game like Anagram, the second player can be a nonhuman player called Non-Playable Character (NPC). A balanced game is more engaging. Therefore, it is imperative to insert artificial intelligence (AI) into an NPC to make it possess a balance ability. This study investigates the AI algorithm that is the most appropriate to make a balance NPC for Anagram games. We tested three scenarios: Descending AI, Random AI, and AI with k-Nearest Neighbour (k-NN). Descending AI gets an Anagram solution by selecting a word with the highest score from all possible answers. Random AI picks a word randomly from the possible answers, while AI with k-NN chooses a word closest to one of the human players. The results show that Descending AI is the best algorithm to make the strongest NPC, which always gets the highest score, followed by Random AI and AI with k-NN. However, AI with the k-NN algorithm makes the constructed NPC has the highest number of turns at an average of 18, while Descending AI gets 14 turns and Random AI has 15 turns. Looking at the remaining lives at the end of the game, AI with k-NN makes the NPC has 25 lives left, while Descending AI has 59 lives, and Random AI has 48 lives. Less remaining lives suggest that NPC containing AI with the k-NN algorithm matches closer to the human player and therefore is more suitable for Anagram NPC.

Creating the behavior for non-player characters (NPCs) in video games is a complex task that requires the collaboration among programmers and game designers. Usually these game designers are responsible of configuring and fine tuning certain parameters of the behavior, while programmers write the actual code of those behaviors. That requires several iterations between them. In this paper, we present a new approach for creating the behavior of NPCs that gives more power to the game designer to create behavior without technical knowledge using program by demonstration but preserving the designer confident of the final behavior.

Video games, particularly role-playing games (RPGs), have become narrative and emotional spaces where players not only follow complex plots but also form affective bonds with non-player characters (NPCs). This article analyses such parasocial relationships through two perspectives: the Affectivity System (AS), which structures the bond between player and NPC, and the model of emotions in video games, which distinguishes among narrative (Fictional), aesthetic (Artefact), and interactive (Gameplay) emotions. The focus is on Clair Obscur: Expedition 33 (Sandfall Interactive, 2025), a European JRPG that integrates mechanics designed to build deep emotional relationships with NPCs. Here, affective interaction is not a byproduct but a central component of the design. The article opens with a reflection on the concept of para-sociality, showing how the interactivity of video games transforms the player into an active agent in the relationship. In this context, the ability to recognize emotions and intentions in virtual characters triggers genuine empathetic responses, making the experience deeply engaging.

This paper firstly seeks an answer to the question of whether Non Player Characters (NPCs) in Computer Games can also be viewed as game agents where reactivity, autonomy, being temporarily continuous and having goal-oriented behavior are taken as the features of being a game agent. Within this scope, we will try to assess whether naming NPCs as agents point to a desire that one day they would fulfill the requirements of being an agent or whether NPCs actually already fulfill these requirements. Secondly, the paper looks into the AI needs of video games. We present the AI methodologies that are either being used and under research for use in Game AI. The same methodologies are also likely to contribute to the increasing levels of autonomy and goal-directed behavior of NPCs and help them become more agent-like.

Non-Player Characters (NPCs) are critical to the interactive and dynamic character of modern video games. They inhabit game worlds, engage with players, and play a large role in storytelling and gameplay mechanics. Yet, conventional rule-based NPCs tend to fall short of realism, flexibility, and scalability needed to keep pace with changing player expectations and sophisticated game worlds. This paper discusses the intersection of Artificial Intelligence (AI) engineering techniques with game development to create intelligent, adaptive NPCs. Through modular AI architectures, reinforcement learning, behaviour cloning, and effective deployment strategies, this research presents a complete framework for developing intelligent NPCs that provide increased interactivity and engagement. It also discusses the application of memory repositories to preserve NPC contextual awareness across extended interactions. Comparative analysis and case studies are employed to illustrate enhancements in gameplay dynamics and player experience.

Nowadays, the video gaming experience is shifting from merely realistic to believable. The behaviour of the computer driven player and non-playing characters is often poor when compared to their visual appearance. In this sense, there has been a recent interest in improving the video gaming experience with novel Artificial Intelligence (AI) techniques. This paper presents a robotics inspired behavioural AI to simulate characters' personalities in a commercial video game.

Non-Playable Characters (NPCs) are a subtype of virtual agents that populate video games by endorsing social roles in the narrative. To infer NPCs’ roles, players evaluate NPCs’ appearance and behaviors, usually by ascribing human traits to NPCs, such as intelligence, likability and morality. In particular, hostile NPCs in video games are essential to build the games’ inherent challenges. The three experiments reported here investigated the extent to which the perception of hostility in a military shooter game (including both threat of appearance and aggressiveness in behaviors) is influenced by the appearance and the behaviors of NPCs thanks to perceived intelligence, likability and morality-related questionnaires. Our results first show that hostility is efficiently conveyed through NPCs’ behaviors, but not significantly by their appearance. Second, our study allows identifying the main predictors of hostility perception, namely unfriendliness, knowledge and harmfulness.

Nowadays, the video gaming experience is shifting from merely realistic to believable. The increasing graphic power of current graphic cards has made it possible to render near lifelike images. Unfortunately, the behaviour of the computer driven player and non-playing characters is often poor when compared to their visual appearance. In this sense, there has been a recent interest in improving the video gaming experience with novel Artificial Intelligence (AI) techniques. This paper presents a robotics inspired behavioural AI technique to simulate characters' personalities in an multi-award winning commercial video game.

This study explores the application of generative artificial intelligence (AI) in video games, focusing on improving content richness and personalized player experience. Traditional non-player character (NPC) systems have limitations in adaptability and interactivity. This study explores how generative AI can improve NPC behavior and dialogue. Then, this paper proposes a framework that combines generative agents, Transformer-based dialogue models, and reinforcement learning. Specifically, the generative agent simulates memory-driven planning, the Transformer model generates context-aware dialogues, and reinforcement learning supports adaptive interactions. This study is based on a generative agent dataset and a Role-playing game (RPG) dialogue corpus. The results show that the proposed method enhances the realism of NPCs, the coherence of game narratives, and the responsiveness of player interactions, and improves player immersion and the diversity of interactions. This provides practical insights for scalable intelligent game development and shows the potential of artificial intelligence in automating complex content creation and points out the direction for the future combination of games and artificial intelligence.

This paper explores the issue of male dominance in the video game industry and gaming community. Female game characters in video games are often over-sexualized to suit male preferences due to mainstream social consciousness. This practice not only disregards the needs and experiences of female gamers, but also reinforces gender stereotypes and hurts gender equality. This kind of disrespect and harm to women further spreads to real social life, which is not conducive to women’s physical and mental health. The paper analyses how male hegemony is manifested through the marginalization of women, the distortion of women’s representation, and the objectification of women as objects of male desire from the perspective of social psychology. In addition, the paper explores how gaming mechanics control women in games and real life and further exacerbate this problem by creating ‘second selves’ that perpetuate gender norms. The paper argues for the need to challenge this male dominance and discusses how women can actively resist this dominance in society and the gaming community. Finally, the paper calls for a more inclusive and gender-equitable gaming culture to improve the gaming experience for all players, as well as to promote respect for women in society.