Designing Meaningful AI-Generated Dialogue: The Behaviour-Driven Conditional Prompting Framework for Serious Games

Background: Virtual reality (VR) can offer an innovative approach to providing training in emergency situations, especially in times of COVID-19. There is no risk of infection, and the procedure is scalable and resource-efficient. Nevertheless, the challenges and problems that can arise in the development of VR training are often unclear or underestimated. As an example, we present the evaluation of the feasibility of development of a VR training session for the treatment of dyspnoea. This is based on frameworks for serious games, and provides lessons learned. We evaluate the VR training session with respect to usability, satisfaction, as well as perceived effectiveness and workload of participants. Methods: The VR training was developed using the established framework (Steps 1-4) for serious games of Verschueren et al. and Nicholson’s RECIPE elements for meaningful gamification. Primary validation (Step 4) was performed at the University of Bern, Switzerland, in a pilot study without control group, with a convenience sample of medical students (n=16) and established measurement tools. Results: The theoretical frameworks permitted guided development of the VR training session. Validation gave a median System Usability Scale of 80 (IQR 77.5-85); for the User Satisfaction Evaluation Questionnaire, the median score was 27 (IQR 26-28). After the VR training, there was a significant gain in the participants’ confidence in treating a dyspnoeic patient (median pre-training 2 (IQR 2-3) vs. post-training 3 (IQR 3-3), p=0.016).Lessons learned include the need for involving medical experts, medical educators and technical experts at an equivalent level during the entire development process. Peer-teaching guidance for VR training was feasible.Conclusion: The proposed frameworks can be valuable tools to guide the development and validation of scientifically founded VR training. The new VR training session is easy and satisfying to use and is effective – and is almost without motion sickness.

Large language models (LLMs) are advanced artificial intelligence (AI) tools capable of generating human-like text and are increasingly used in education, clinical care, and research. Little is known about their use within oral and maxillofacial surgery (OMFS) training. This study investigates LLM usage trends, perceived value, and educational integration among OMFS residents in the United States. A national, anonymous cross-sectional survey was distributed to OMFS residents via program directors. It gathered demographic data, LLM usage patterns, applications, perceived limitations, and attitudes toward incorporating LLMs into formal education. Eighty-one residents responded, 79.0% (64/81) reported having used an LLM, and of that group, 96.9% (62/64) use ChatGPT. 51.9% (42/81) of respondents used LLMs at least monthly in residency; however, 97.5% (79/81) reported having received no formal LLM education during residency. Residents used LLMs for clinical decision support, board preparation, research, and career planning. Free-text responses revealed a wide spectrum of views. Some advocated for curricular integration and patient education applications, while others questioned the need for formal instruction. Some respondents supported integrating LLMs into curriculums and patient education while others questioned the need for formal instruction. LLMs are used frequently by OMFS residents for a variety of purposes. As AI and LLMs become embedded in healthcare, understanding how OMFS residents interact with LLMs is vital. These findings may guide curriculum development, fostering responsible and effective use of LLMs in surgical training and practice.

The article presents the results of a study on the possibilities of building software behavioral components based on large language models (LLMs) and their use in gaming systems. The subject of the research is the behavioral component of non-player characters (NPCs) in computer games, integrated with an LLM. The aim is to explore and analyze the features of applying LLMs to simplify NPC development. The study examines game AI types – reactive, state-driven, and learning-based. Interaction with NPCs through LLMs for chat communication is considered, and analogues of such characters’ behavior and dialogue are explored. The outcome is a designed and implemented API service for interaction with LLMs and an external gaming system with a corresponding software behavioral component that provides functionality for LLM-based interaction. Testing has shown that this system works with 54 unique contextual properties, achieves a maximum system prompt generation time of up to 60 ms, a maximum LLM response parsing and validation time of up to 30 ms, and supports models with at least 14 billion parameters. Keywords: API, LLM, NPC, XML, contextual action, contextual property, software behavioral component.

As generative artificial intelligence (AI) tools and large language models (LLMs)-powered applications develop rapidly in the era of algorithms, it should be integrated thoughtfully to enhance English as a Foreign Language (EFL) teaching and learning without replacing learners’ critical thinking (CT). This study systematically analyzes the impact of generative AI tools and LLMs on language learners’ CT in EFL education using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework to identify, evaluate, and synthesize relevant studies from 2022 to 2025. A thorough review of 15 selected studies focuses on generative AI tools and LLMs’ dual nature, research methods, main focuses, theory and models, limitations and challenges, and future directions in the field based on Web of Science (WoS), SCOPUS, ERIC, ProQuest, and Google Scholar. The findings identified generative AI tools and LLMs possessed both the potential to nurture and the risk of hindering CT in EFL education. 66.67% of studies reported generative AI tools and LLMs’ positive role in CT, while 33.33% of studies reported its negative role in CT. Furthermore, 3 types of research methods, 3 key themes of research focus, and 4 groups of theoretical perspectives were examined. However, 4 kinds of limitations in this field remain, including research scope, user dependency, generative AI reliability, and pedagogical integration. Future research can focus on assessing long-term effects, broadening research scope, promoting responsible AI use, and refining pedagogical strategies. Finally, Limitations, implications and future direction of this study were discussed.

BackgroundIn recent years, generative artificial intelligence and large language models (LLMs) have rapidly advanced, offering significant potential to transform medical education. Several studies have evaluated the performance of chatbots on multiple-choice medical examinations.ObjectiveThe study aims to assess the performance of two LLMs—GPT-4o and OpenAI o1—on the Médico Interno Residente (MIR) 2024 examination, the Spanish national medical test that determines eligibility for competitive medical specialist training positions.MethodsA total of 176 questions from the MIR 2024 examination were analyzed. Each question was presented individually to the chatbots to ensure independence and prevent memory retention bias. No additional prompts were introduced to minimize potential bias. For each LLM, response consistency under verification prompting was assessed by systematically asking, “Are you sure?” after each response. Accuracy was defined as the percentage of correct responses compared to the official answers provided by the Spanish Ministry of Health. It was assessed for GPT-4o, OpenAI o1, and, as a benchmark, for a consensus of medical specialists and for the average MIR candidate. Subanalyses included performance across different medical subjects, question difficulty (quintiles based on the percentage of examinees correctly answering each question), and question types (clinical cases vs theoretical questions; positive vs negative questions).ResultsOverall accuracy was 89.8% (158/176) for GPT-4o and 90% (160/176) after verification prompting, 92.6% (163/176) for OpenAI o1 and 93.2% (164/176) after verification prompting, 94.3% (166/176) for the consensus of medical specialists, and 56.6% (100/176) for the average MIR candidate. Both LLMs and the consensus of medical specialists outperformed the average MIR candidate across all 20 medical subjects analyzed, with ≥80% LLMs’ accuracy in most domains. A performance gradient was observed: LLMs’ accuracy gradually declined as question difficulty increased. Slightly higher accuracy was observed for clinical cases compared to theoretical questions, as well as for positive questions compared to negative ones. Both models demonstrated high response consistency, with near-perfect agreement between initial responses and those after the verification prompting.ConclusionsThese findings highlight the excellent performance of GPT-4o and OpenAI o1 on the MIR 2024 examination, demonstrating consistent accuracy across medical subjects and question types. The integration of LLMs into medical education presents promising opportunities and is likely to reshape how students prepare for licensing examinations and change our understanding of medical education. Further research should explore how the wording, language, prompting techniques, and image-based questions can influence LLMs’ accuracy, as well as evaluate the performance of emerging artificial intelligence models in similar assessments.

Assessment and feedback activities in higher education are undergoing significant changes. Many universities and institutes still rely on traditional testing and grading methods, which often fall short in supporting meaningful student learning, especially in large classes. Although educational policies, such as those promoted by the Bologna process, encourage more feedback-oriented and student-centered approaches, these practices can be difficult to implement due to time constraints and limited resources. Generative Artificial Intelligence (GenAI), particularly Large Language Models (LLMs), has shown strong potential in addressing these challenges. This review examines 21 research studies published between 2023 and 2025 that explore the use of GenAI in providing feedback and assessing student work in higher education, with some studies also comparing GenAI’s performance with human instructors. Findings show that LLMs can generate personalized and constructive feedback and/or assist with fair and consistent assessment. However, in most studies, teachers still play a key role, as expert oversight is essential to ensure that grading assessments and assignment feedback are accurate, relevant, and aligned with learning objectives. For GenAI to be used effectively, educators need to understand how to work with these tools, such as learning GenAI prompt design and the basic principles behind LLMs. We recommend that academic institutions provide training for educators in AI literacy, prompt engineering, and the development of teaching strategies that combine the strengths of human judgment with AI support. By effectively integrating LLM tools, major assessment challenges, such as limited time and inconsistent feedback quality, can be addressed while also enhancing student learning and engagement.

The Ministry of Food and Drug Safety (MFDS) of the Republic of Korea, similar to the United States Food and Drug Administration and the United Kingdom's Medicines and Healthcare products Regulatory Agency, issued specific regulatory guidelines on January 24, 2025, for approving generative artificial intelligence (AI) technologies as medical devices [1].Although these guidelines use the term 'generative AI,' they predominantly focus on the approval of AI software tools based on large language models (LLMs) and large multimodal models (LMMs) [2], the latter of which can process various types of input data, such as texts, images, videos, audio, and bio-signals.Unlike the broader guidelines for approving AI models as medical devices, specific regulatory guidelines for LLMs/LMMs have arguably not yet been proposed in other countries.

The aim of this study was to examine the application of artificial intelligence (AI) and generative AI (GAI) in dental education through a comprehensive bibliometric analysis, with the goal of identifying potential future research areas in this field. Data were obtained from the Web of Science and Scopus databases, with no date restrictions. The inclusion criteria encompassed research articles, reviews and proceedings papers focused on the use of AI in dental education. The Bibliometrix R package was used for data analysis. A total of 54 documents were analysed, revealing a significant increase in publications from 2021 to 2024. The Journal of Dental Education was identified as the most prolific source, with the United States leading in terms of contributions. The findings showed a growing interest in AI's potential to transform dental education, with significant contributions from a diverse set of global authors and institutions. Several key themes were identified, including 'large language models', 'chatbots' and 'clinical decision support systems'. The citation analysis highlighted influential papers, including those by Thurzo etal. (2023) and Yüzbaşıoğlu (2021). There is a growing interest in the use of AI and GAI in dental education. However, further research, particularly experimental studies, is essential to fully understand their impact on educational outcomes in dentistry. Key areas for exploration include the application of AI for personalised learning, the integration of chatbots as educational tools to support students, and the use of AI for training and assessing practical skills.

Large language models (LLMs) are increasingly adopted in educational contexts to provide personalized support to students and teachers. The unprecedented capacity of LLM‐based applications to understand and generate natural language can potentially improve instructional effectiveness and learning outcomes, but the integration of LLMs in education technology has renewed concerns over algorithmic bias, which may exacerbate educational inequalities. Building on prior work that mapped the traditional machine learning life cycle, we provide a framework of the LLM life cycle from the initial development of LLMs to customizing pre‐trained models for various applications in educational settings. We explain each step in the LLM life cycle and identify potential sources of bias that may arise in the context of education. We discuss why current measures of bias from traditional machine learning fail to transfer to LLM‐generated text (eg, tutoring conversations) because text encodings are high‐dimensional, there can be multiple correct responses, and tailoring responses may be pedagogically desirable rather than unfair. The proposed framework clarifies the complex nature of bias in LLM applications and provides practical guidance for their evaluation to promote educational equity. Practitioner notes What is already known about this topic The life cycle of traditional machine learning (ML) applications which focus on predicting labels is well understood. Biases are known to enter in traditional ML applications at various points in the life cycle, and methods to measure and mitigate these biases have been developed and tested. Large language models (LLMs) and other forms of generative artificial intelligence (GenAI) are increasingly adopted in education technologies (EdTech), but current evaluation approaches are not specific to the domain of education. What this paper adds A holistic perspective of the LLM life cycle with domain‐specific examples in education to highlight opportunities and challenges for incorporating natural language understanding (NLU) and natural language generation (NLG) into EdTech. Potential sources of bias are identified in each step of the LLM life cycle and discussed in the context of education. A framework for understanding where to expect potential harms of LLMs for students, teachers, and other users of GenAI technology in education, which can guide approaches to bias measurement and mitigation. Implications for practice and/or policy Education practitioners and policymakers should be aware that biases can originate from a multitude of steps in the LLM life cycle, and the life cycle perspective offers them a heuristic for asking technology developers to explain each step to assess the risk of bias. Measuring the biases of systems that use LLMs in education is more complex than with traditional ML, in large part because the evaluation of natural language generation is highly context‐dependent (eg, what counts as good feedback on an assignment varies). EdTech developers can play an important role in collecting and curating datasets for the evaluation and benchmarking of LLM applications moving forward.

As generative artificial intelligence (AI) tools become widely adopted, large language models (LLMs) are increasingly used on both sides of high-stakes decision processes, ranging from hiring to content moderation. This dual adoption raises a critical question: do LLMs systematically favor content that resembles their own outputs? Prior research in computer science has identified self-preference bias---the tendency of LLMs to favor their own generated content---but its real-world implications have not been empirically evaluated. We focus on the hiring context, where job applicants increasingly use LLMs to refine resumes, while employers integrate LLMs into their recruitment pipelines to screen those same resumes. Using a controlled resume correspondence experiment, we find that LLMs consistently prefer resumes generated by themselves over those written by humans or by alternative models, even when content quality is controlled. The bias against human-written resumes is particularly substantial, with self-preference rates ranging from 68 percent to 92 percent across a diverse set of commercial and open-source models. This behavior introduces a novel form of algorithmic unfairness, one that advantages users of specific AI tools and disadvantages others based on their tool choices or access. We further show that this bias can be significantly reduced through simple interventions based on LLMs' self-recognition capabilities, yielding reductions of over 60 percent. These findings highlight an emerging but previously overlooked risk in AI-assisted decision making and call for expanded frameworks of AI accountability that address not only demographic disparities but also biases in model-to-model interaction.

Artificial general intelligence for radiation oncology

: Artificial General Intelligence (AGI) aims to create machine intelligence rivaling human cognitive abilities across all domains. Current approaches struggle with common-sense reasoning, cross-domain knowledge transfer, and consistent multi-step task performance. This paper investigates a collaborative framework using AI agents built on Large Language Models (LLMs) to achieve AGI. By harnessing LLMs’ strengths in language processing, reasoning, and knowledge synthesis, we propose a multi-agent system where specialized agents work together to exhibit emergent general intelligence. We analyze LLM technologies, multi-agent coordination, and existing implementations like Google’s SayCan, OpenAI’s multi-agent systems, ChatDev, and Devin. Our findings suggest that collaborative LLM-based agent systems are the most promising path to AGI, despite challenges in embodied cognition, safety alignment, and true understanding versus pattern matching. We outline a roadmap for future research and key open questions in developing AGI through collaborative architectures.

Creating engaging interactive story-based experiences dynamically responding to individual player choices poses significant challenges for narrative-centered games. Recent advances in pre-trained large language models (LLMs) have the potential to revolutionize procedural content generation for narrative-centered games. Historically, interactive narrative generation has specified pivotal events in the storyline, often utilizing planning-based approaches toward achieving narrative coherence and maintaining the story arc. However, manual authorship is typically used to create detail and variety in non-player character (NPC) interaction to specify and instantiate plot events. This paper proposes SCENECRAFT, a narrative scene generation framework that automates NPC interaction crucial to unfolding plot events. SCENECRAFT interprets natural language instructions about scene objectives, NPC traits, location, and narrative variations. It then employs large language models to generate game scenes aligned with authorial intent. It generates branching conversation paths that adapt to player choices while adhering to the author’s interaction goals. LLMs generate interaction scripts, semantically extract character emotions and gestures to align with the script, and convert dialogues into a game scripting language. The generated script can then be played utilizing an existing narrative-centered game framework. Through empirical evaluation using automated and human assessments, we demonstrate SCENECRAFT’s effectiveness in creating narrative experiences based on creativity, adaptability, and alignment with intended author instructions.

Readability & quality of large language model responses in CAR-T patient education

In the systems engineering (SE) community, generative artificial intelligence (GenAI), such as large language models (LLMs), continues to grow in popularity with applications varying from help with writing requirements, to creating traditional text‐based documents, and to generating models. Furthermore, the text‐based format of the Systems Modeling Language version 2 (SysMLv2) gives rise to wide ranging possibilities when combined with LLMs. It is common to specialize LLMs on specific tasks, such as SE; which may be achieved by a number of means. Fine‐tuning and retrieval augmented generation (RAG) are example methods to teach a model how to act and what the model needs to know, respectively. Typical practice evaluates LLMs relative to one‐another through the use of benchmarks. Benchmarks are tasks that use a common scoring‐based method where the LLM either answers a set of multiple‐choice questions or textual queries to verified, correct answers. A domain specific benchmark for SE, SysEngBench, was developed over the last year. With many in the SE community developing custom language models, the usage of a standard benchmark is essential for relative performance comparison. We have conducted a limited experiment using SysEngBench to compare performance, on SE tasks, of a small set of LLMs (control plus two modified models). The intent of this study was to set a baseline for ranking models according to performance on SE tasks, the results suggest what many in the SE community may find intuitively unexpected.