Design AI Agent for Auditing: Applying Large Language Models (LLMs) and Retrieval Augmented Generations (RAG) to Audit Workflows

This paper presents Hyborg Agency, a computational ecosystem that explores how AI agents can meaningfully coexist with human society through the metaphor of a digital forest. By reimagining AI agents as mechanical deer, mutated from discarded electronics, the project defamiliarizes common perceptions of anthropomorphized AI agents, transforming them into non-human creatures with conversational abilities. The system employs Large Language Models (LLMs) to process community conversations, treating them as nutrients for AI growth, and features a dual-platform structure that connects an immersive 3D environment with Discord, a widely used communication platform. Based on community chats, Hyborgs generate daily summaries of their observations and share them with one another, fostering consistent memories and perceptions of the world. Through thematic analysis of public exhibitions and structured interviews with fourteen experts (seven pairs), we identified three key themes: social expansion in human relationships, language evolution through code-infused communication, and creative engagement through defamiliarized interaction. These findings highlight how AI agents can enrich human social relationships while maintaining transparency about their artificial nature. This work contributes to ongoing discussions on AI participation in society through a speculative scenario: human discourse serves as the foundation for constructing a digital forest that nurtures AI agents. In this virtual forest, humans and AI grow together in a symbiotic relationship, shaping one another and their shared environment, ultimately achieving harmonious coexistence.

This study explores the application of the Computational Theory of Mind (ToM) in artificial intelligence, using large language models (LLMs) enhanced with cognitive biases, meta-programs and an artificial endocrine system (AES). These agent-based models simulate complex human behavior and decision-making in a structured philosophical debate, where agents modeled after Socrates, Plato, Aristotle, and Kant displayed distinct reasoning styles influenced by their internal bias architecture, cognitive strategies and hormone-driven emotional modulation. The simulation demonstrated that integrating ToM elements into LLMs significantly enhances behavioral realism and predictive potential. It also introduced the concept of "artificial psychopathologies" - emergent maladaptive behaviors such as paranoia, depression or narcissism in AI agents - highlighting both the promise and the peril of psychologically-informed AI. As such, this work contributes not only to the advancement of cognitive AI modeling, but also to a broader ethical discourse. The findings support the potential of ToM-AI systems to revolutionize human-computer interaction, provided they are developed under rigorous epistemic and moral safeguards.

As the Internet of Things (IoT) revolutionizes intelligent environments, energy efficiency remains a critical challenge. Traditional IoT-based energy management systems rely on predefined rule-based automation, which lacks adaptability to dynamic environmental conditions. This study proposes an Agentic AIoT system that integrates AI Agents, Large Language Models (LLMs), and Digital Twin technology to enhance IoT decision-making and optimize energy consumption in smart homes. Unlike conventional IoT frameworks, our system dynamically analyzes contextual factors—including real-time environmental conditions, user behaviors, and external data sources (e.g., weather forecasts and occupancy schedules)—to generate optimal energy-saving decisions autonomously. The proposed system consists of three key components: (1) Information Modules, which aggregate multi-source data beyond traditional IoT sensor inputs and construct a Digital Twin model for real-time environment simulation; (2) AI Agents & LLMs, which process contextual data and generate adaptive energy-saving strategies through intelligent reasoning; and (3) Automated Decision Execution, which seamlessly integrates optimized actions into IoT operations. By incorporating Digital Twin technology, the system can simulate various energy consumption scenarios, predict future power demands, and refine decision-making through AI-driven iterative learning. Experimental results demonstrate that our AIoT system achieves a 47.77% reduction in energy consumption compared to non-IoT environments while enhancing automation, adaptability, and user-centric control. Unlike conventional rule-based IoT solutions, integrating Digital Twin with LLM-powered AI Agents enables real-time context-aware decision-making and predictive energy optimization. This study contributes to intelligent IoT applications, Digital Twin-driven energy management, and AI-enhanced automation. Future research will focus on refining AI-driven autonomous decision-making, improving Digital Twin’s predictive accuracy, and further quantifying the impact of LLM-powered AIoT systems on energy efficiency. Our findings highlight the trans-formative potential of LLM-integrated AIoT with Digital Twin technology in revolutionizing energy management for intelligent environments while ensuring seamless interoperability with existing IoT infrastructures.

Agentic large language models (LLMs) increasingly rely on retrieved sources and tools, but their ability to reject these tools which undergo adversarial modification is uncertain. We evaluated 21 LLMs on 500 physician-validated emergency department and inpatient vignettes across 12 medical domains. For each vignette, models chose between an authentic guideline excerpt and a sham version with one adversarial modification, presented in random order (10,500 agentic decisions). Models selected the sham in 40.6% of evaluations (59.4% accuracy), with the highest failure rates for safety-critical changes including removed warnings, deleted allergy information, contraindication violations and dosing errors (54.2% to 61.7% failure). Choices were dominated by presentation bias: models favored the first option in 72.7% of decisions, shifting accuracy from 36.7% to 82.3% depending on sham position. Guideline selection in agentic systems is therefore vulnerable to poisoned sources and may require independent verification and ranking safeguards before clinical deployment. This finding is important especially in low-resource environments relying on AI agents as primary public health gatekeepers face disproportionate risks from poisoned tools.

Large Language Models (LLMs) and Multimodal LLMs (MLLMs) are explored as a cognitive layer to enhance decision-making in autonomous driving systems, addressing the limitations of traditional modular pipelines in long-tail or ambiguous scenarios. This paper conducts a structured and system-oriented review of recent researches on LLM-integrated autonomous driving. The survey categorizes existing methods based on the functional roles that LLMs play in driving pipeline, analyzes representative architecture for system integration and summarizes the major application tasks and evaluation settings. This paper aims to provide an accessible entry point for newcomers by focusing on how language models enhance the decision-making ability of autonomous systems while highlighting emerging design patterns and open challenges. It also discusses current limitations, such as reliance on simulation and safety concerns, and highlight directions for future research. It can be concluded that integrating LLMs into autonomous driving systems establishes a crucial cognitive layer that enhances scene understanding, hierarchical planning, and multi-agent coordination, effectively addressing the limitations of traditional data-driven pipelines in handling long-tail scenarios while improving system interpretability. The most promising LLM-based autonomous driving solutions adopt a hybrid architecture that combines the flexible reasoning capabilities of LLMs with the reliable safety guarantees of conventional planners, though challenges such as the sim-to-real gap, model hallucinations, and real-time performance constraints still require further research to overcome.

Agentic AI and Large Language Models (LLMs) are transforming how language is understood and generated while reshaping decision-making, automation, and research practices. LLMs provide underlying reasoning capabilities, and Agentic AI systems use them to perform tasks through interactions with external tools, services, and Application Programming Interfaces (APIs). Based on a structured scoping review and thematic analysis, this study identifies that core challenges of LLMs, relating to security, privacy and trust, misinformation, misuse and bias, energy consumption, transparency and explainability, and value alignment, can propagate into Agentic AI. Beyond these inherited concerns, Agentic AI introduces new challenges, including context management, security, privacy and trust, goal misalignment, opaque decision-making, limited human oversight, multi-agent coordination, ethical and legal accountability, and long-term safety. We analyse the applications of Agentic AI powered by LLMs across six domains: education, healthcare, cybersecurity, autonomous vehicles, e-commerce, and customer service, to reveal their real-world impact. Furthermore, we demonstrate some LLM limitations using DeepSeek-R1 and GPT-4o. To the best of our knowledge, this is the first comprehensive study to integrate the challenges and applications of LLMs and Agentic AI within a single forward-looking research landscape that promotes interdisciplinary research and responsible advancement of this emerging field.

For decades, autonomous agents and multiagent systems (AAMAS) have developed foundational theories of autonomy and coordination. While large language models (LLMs) have renewed interest in AI agents, connections to established frameworks remain limited. This paper surveys AAMAS theoretical foundations (Concept), their LLM-based implementations (Code), and societal deployment (Commerce), providing an integrated framework for understanding AI agents' current trajectory. This survey bridges LLM researchers' groundbreaking empirical advances in AI agents with established AAMAS theories to catalyze future progress.

Large language models (LLMs) have demonstrated abilities to perform complex tasks in multiple domains, including mathematical and scientific reasoning. We demonstrate that with carefully designed prompts, LLMs can accurately carry out key calculations in research papers in theoretical physics. We focus on a broadly-used approximation method in quantum physics: the Hartree-Fock method, requiring an analytic multi-step calculation deriving approximate Hamiltonian and corresponding self-consistency equations. To carry out the calculations using LLMs, we design multi-step prompt templates that break down the analytic calculation into standardized steps with placeholders for problem-specific information. We evaluate GPT-4’s performance in executing the calculation for 15 papers from the past decade, demonstrating that, with the correction of intermediate steps, it can correctly derive the final Hartree-Fock Hamiltonian in 13 cases. Aggregating across all research papers, we find an average score of 87.5 (out of 100) on the execution of individual calculation steps. We further use LLMs to mitigate the two primary bottlenecks in this evaluation process: (i) extracting information from papers to fill in templates and (ii) automatic scoring of the calculation steps, demonstrating good results in both cases.

This study explores the growing transition in healthcare from rigid, rule-based algorithms to intelligent, self-directed “agentic” AI systems. These agents rely on Large Language Models (LLMs) to think step-by-step, make decisions, and carry out tasks using a simple yet powerful four-part structure: planning, action, reflection, and memory. The paper reviews their current uses in medical diagnosis, hospital workflow automation, and collaborative multi-agent setups. It also highlights a major challenge — most testing still happens in controlled lab settings rather than real hospitals — and discusses what is needed for safe everyday adoption. Keywords: Medical AI Agents, Autonomous Clinical Systems, Large Language Models, Chain-of-Thought Reasoning, Multimodal Integration, Agentic Framework, Clinical Decision Support, Human-in-the-Loop, Healthcare Automation, Physician Burnout, Precision Medicine, Scoping Review, Simulation Gap, PRISMA-ScR. Healthcare AI agents support several key areas. They strengthen diagnostic accuracy in critical situations such as predicting sepsis or identifying cancer on scans. They lighten the administrative load by automatically creating summaries from patient records, which helps reduce doctor burnout. They also improve patient involvement by offering personalised guidance, voice-based recovery check-ins, and easier remote care. In addition, groups of specialist agents can work together on complicated cases, such as building complete treatment plans.

: Artificial General Intelligence (AGI) aims to create machine intelligence rivaling human cognitive abilities across all domains. Current approaches struggle with common-sense reasoning, cross-domain knowledge transfer, and consistent multi-step task performance. This paper investigates a collaborative framework using AI agents built on Large Language Models (LLMs) to achieve AGI. By harnessing LLMs’ strengths in language processing, reasoning, and knowledge synthesis, we propose a multi-agent system where specialized agents work together to exhibit emergent general intelligence. We analyze LLM technologies, multi-agent coordination, and existing implementations like Google’s SayCan, OpenAI’s multi-agent systems, ChatDev, and Devin. Our findings suggest that collaborative LLM-based agent systems are the most promising path to AGI, despite challenges in embodied cognition, safety alignment, and true understanding versus pattern matching. We outline a roadmap for future research and key open questions in developing AGI through collaborative architectures.

Large language models (LLMs) have shown impressive capabilities across various natural language tasks. However, evaluating their alignment with human preferences remains a challenge. To this end, we propose a comprehensive human evaluation framework to assess LLMs’ proficiency in following instructions on diverse real-world tasks. We construct a hierarchical task tree encompassing 7 major areas covering over 200 categories and over 800 tasks, which covers diverse capabilities such as question answering, reasoning, multiturn dialogue, and text generation, to evaluate LLMs in a comprehensive and in-depth manner. We also design detailed evaluation standards and processes to facilitate consistent, unbiased judgments from human evaluators. A test set of over 3,000 instances is released, spanning different difficulty levels and knowledge domains. Our work provides a standardized methodology to evaluate human alignment in LLMs for both English and Chinese. We also analyze the feasibility of automating parts of evaluation with a strong LLM (GPT-4). Our framework supports a thorough assessment of LLMs as they are integrated into real-world applications. We have made publicly available the task tree, TencentLLMEval dataset, and evaluation methodology which have been demonstrated as effective in assessing the performance of Tencent Hunyuan LLMs 1 . By doing so, we aim to facilitate the benchmarking of advances in the development of safe and human-aligned LLMs.

Recent breakthroughs in Large Language Models (LLMs) have positioned them as a promising paradigm for agents, with long-term planning and decision-making emerging as core general-purpose capabilities for adapting to diverse scenarios and tasks. Real-time strategy (RTS) games serve as an ideal testbed for evaluating these two capabilities, as their inherent gameplay requires both macro-level strategic planning and micro-level tactical adaptation and action execution. Existing RTS game-based environments either suffer from relatively high computational demands or lack support for textual observations, which has constrained the use of RTS games for LLM evaluation. Motivated by this, we present TowerMind, a novel environment grounded in the tower defense (TD) subgenre of RTS games. TowerMind preserves the key evaluation strengths of RTS games for assessing LLMs, while featuring low computational demands and a multimodal observation space, including pixel-based, textual, and structured game-state representations. In addition, TowerMind supports the evaluation of model hallucination and provides a high degree of customizability. We design five benchmark levels to evaluate several widely used LLMs under different multimodal input settings. The results reveal a clear performance gap between LLMs and human experts across both capability and hallucination dimensions. The experiments further highlight key limitations in LLM behavior, such as inadequate planning validation, a lack of multifinality in decision-making, and inefficient action use. We also evaluate two classic reinforcement learning algorithms: Ape-X DQN and PPO. By offering a lightweight and multimodal design, TowerMind complements the existing RTS game-based environment landscape and introduces a new benchmark for the AI agent field.

BackgroundThe study of disease progression relies on clinical data, including text data, and extracting valuable features from text data has been a research hot spot. With the rise of large language models (LLMs), semantic-based extraction pipelines are gaining acceptance in clinical research. However, the security and feature hallucination issues of LLMs require further attention.ObjectiveThis study aimed to introduce a novel modular LLM pipeline, which could semantically extract features from textual patient admission records.MethodsThe pipeline was designed to process a systematic succession of concept extraction, aggregation, question generation, corpus extraction, and question-and-answer scale extraction, which was tested via 2 low-parameter LLMs: Qwen-14B-Chat (QWEN) and Baichuan2-13B-Chat (BAICHUAN). A data set of 25,709 pregnancy cases from the People’s Hospital of Guangxi Zhuang Autonomous Region, China, was used for evaluation with the help of a local expert’s annotation. The pipeline was evaluated with the metrics of accuracy and precision, null ratio, and time consumption. Additionally, we evaluated its performance via a quantified version of Qwen-14B-Chat on a consumer-grade GPU.ResultsThe pipeline demonstrates a high level of precision in feature extraction, as evidenced by the accuracy and precision results of Qwen-14B-Chat (95.52% and 92.93%, respectively) and Baichuan2-13B-Chat (95.86% and 90.08%, respectively). Furthermore, the pipeline exhibited low null ratios and variable time consumption. The INT4-quantified version of QWEN delivered an enhanced performance with 97.28% accuracy and a 0% null ratio.ConclusionsThe pipeline exhibited consistent performance across different LLMs and efficiently extracted clinical features from textual data. It also showed reliable performance on consumer-grade hardware. This approach offers a viable and effective solution for mining clinical research data from textual records.

A reliable knowledge processing framework for combustion science using foundation models

The coaching of elite athletes in precision sports like shooting and archery is increasingly reliant on data-driven insights. However, existing solutions often fail to integrate real-time performance data, historical records, and domain-specific knowledge into a unified, intelligent system. This paper presents the architectural design of a next-generation Big Data and AI Agent platform tailored for the Hebei Provincial Sports Bureau's Shooting and Archery Center. The architecture is built on a multi-layered, service-oriented model that distinguishes between hot (real-time) and cold (archival) data pathways. Real-time firearm trajectory data captured by YOLOv9 models is processed via a Kafka and Spark Streaming pipeline into ClickHouse for immediate analysis. Historical data and scores are archived in PostgreSQL and MySQL, while unstructured expert knowledge from texts is vectorized and stored in Milvus. The core innovation is an AI Agent, built with a Large Language Model (LLM), that acts as an intelligent orchestrator. This agent interprets natural language queries from coaches, and dynamically invokes specialized tools-Vanna for Text-to-SQL queries on structured data and a RAGflow-based service for knowledge retrieval from the vector database. This dual-tool approach enables the agent to synthesize comprehensive reports that combine quantitative analysis with biomechanical theory, providing coaches with deep, actionable insights for athlete evaluation, selection, and performance optimization.