An enhanced reasoning framework with closed-loop refinement for reliable LLM-driven drone control

A previous study investigated the application of Large Language Models (LLM) in Agent-Based Simulations for System of Systems studies, where the LLM acted as an Incident Commander during a wildfire event. Firefighting Concept of Operations were utilized as an initial prompt to limit the LLM's answers. The findings were promising, indicating that LLMs may effectively drive simulation agents. Building on these insights, the goal is to integrate an LLM directly into a particular agent's decision-making loop. The present work will investigate whether LLMs perform well in cyber-physical simulation environment. The LLM will be used with a cyber-physical simulation to fly an Unmanned Aerial Vehicle and study its adherence to mission objectives. The LLM's judgments and the computational resources utilized throughout the simulation will be discussed with special interest in future improvements and implications for designing autonomous aerial firefighting systems.

A previous study investigated the application of Large Language Models (LLM) in Agent-Based Simulations for System of Systems studies, where the LLM acted as an Incident Commander during a wildfire event. Firefighting Concept of Operations were utilized as an initial prompt to limit the LLM's answers. The findings were promising, indicating that LLMs may effectively drive simulation agents. Building on these insights, the goal is to integrate an LLM directly into a particular agent's decision-making loop. The present work will investigate whether LLMs perform well in cyber-physical simulation environment. The LLM will be used with a cyber-physical simulation to fly an Unmanned Aerial Vehicle and study its adherence to mission objectives. The LLM's judgments and the computational resources utilized throughout the simulation will be discussed with special interest in future improvements and implications for designing autonomous aerial firefighting systems.

Large language models illuminate a progressive pathway to artificial intelligent healthcare assistant

Large Language Models (LLMs) are rapidly revolutionizing scientific research. These tools, powered by LLMs, are not just assistants for tasks such as literature synthesis, hypothesis generation, pattern identification, and research paper writing, but are evolving into autonomous discovery systems. This review explores the capabilities of these 'AI Scientists' and compares leading entities like Sakana AI's autonomous system and Google's collaborative 'AI co-scientist'. While these advancements promise unprecedented acceleration of scientific progress, they also bring significant ethical challenges, such as bias amplification, reproducibility, transparency, authorship, and equity. By identifying current limitations and analyzing these challenges, we aim to ensure the responsible use of AI in scientific discovery.

The growing interest in advanced large language models (LLMs) like ChatGPT has sparked debate about how best to use them in various human activities. However, a neglected issue in the debate concerning the applications of LLMs is whether they can reason logically and follow rules in novel contexts, which are critical for our understanding and applications of LLMs. To address this knowledge gap, this study investigates five LLMs (ChatGPT-4o, Claude, Gemini, Meta AI, and Mistral) using word ladder puzzles to assess their logical reasoning and rule-adherence capabilities. Our two-phase methodology involves (1) explicit instructions about word ladder puzzles and rules regarding how to solve the puzzles and then evaluate rule understanding, followed by (2) assessing LLMs’ ability to create and solve word ladder puzzles while adhering to rules. Additionally, we test their ability to implicitly recognize and avoid HIPAA privacy rule violations as an example of a real-world scenario. Our findings reveal that LLMs show a persistent lack of logical reasoning and systematically fail to follow puzzle rules. Furthermore, all LLMs except Claude prioritized task completion (text writing) over ethical considerations in the HIPAA test. Our findings expose critical flaws in LLMs’ reasoning and rule-following capabilities, raising concerns about their reliability in critical tasks requiring strict rule-following and logical reasoning. Therefore, we urge caution when integrating LLMs into critical fields and highlight the need for further research into their capabilities and limitations to ensure responsible AI development.

Large language models (LLMs) offer significant promise as a knowledge source for task learning. Prompt engineering has been shown to be effective for eliciting knowledge from an LLM, but alone it is insufficient for acquiring relevant, situationally grounded knowledge for an embodied agent learning novel tasks. We describe a cognitive-agent approach, STARS, that extends and complements prompt engineering, mitigating its limitations and thus enabling an agent to acquire new task knowledge matched to its native language capabilities, embodiment, environment, and user preferences. The STARS approach is to increase the response space of LLMs and deploy general strategies, embedded within the autonomous agent, to evaluate, repair, and select among candidate responses produced by the LLM. We describe the approach and experiments that show how an agent, by retrieving and evaluating a breadth of responses from the LLM, can achieve 77-94% task completion in one-shot learning without user oversight. The approach achieves 100% task completion when human oversight (such as an indication of preference) is provided. Further, the type of oversight largely shifts from explicit, natural language instruction to simple confirmation/discomfirmation of high-quality responses that have been vetted by the agent before presentation to a user.

Large Language Models (LLMs) excel in fields such as natural language understanding, generation, complex reasoning, and biomedicine. With advancements in materials science, traditional manual annotation methods for phase diagrams have become inadequate due to their time-consuming nature and limitations in updating thermodynamic databases. To overcome these challenges, we propose a framework based on instruction tuning, utilizing LLMs for automated end-to-end annotation of phase diagrams. High-quality phase diagram images and expert descriptions are collected from handbooks and then preprocessed to correct errors, remove redundancies, and enhance information. These preprocessed data form a golden dataset, from which a subset are used to train LLMs through hierarchical sampling. The fine-tuned LLM is then tested for automated phase diagram annotation. Results show that the fine-tuned model achieves a cosine similarity of 0.8737, improving phase diagram comprehension accuracy by 7% compared to untuned LLMs. To the best of our knowledge, this is the first paper to propose using LLMs for the automated annotation of phase diagrams, replacing traditional manual annotation methods and significantly enhancing efficiency and accuracy.

e13657 Background: With demanding schedules, oncologists often struggle to stay updated on the latest clinical trial data from publications, especially without access to pre-curated databases. Traditionally, building annotated clinical trial databases required extensive time and manual effort. To address these challenges, we explored assembling an L-SLR library of clinical trial data using extractions generated by an agentic LLM system and evaluated the system’s accuracy and associated time savings. Methods: Agentic LLM systems are autonomous systems where multiple LLMs maintain control over how they accomplish tasks with no human input or supervised training. Our system used three OpenAI LLM models (o1, o3, o1-mini) in a matrix of processes to emulate trained human experts by following an annotation manual, subdividing complex processes into smaller subtasks, and documenting its reasoning for traceable results. This methodology is inspired by the recent development from DeepSeek reinforcement learning and reasoning capabilities. Annotations were created for 4 review variables (population, intervention/comparator, outcome, study design) and 32 extraction variables, including clinical TNM staging, histology, biomarker, associated risk factors, treatment path (line, prior therapy etc.), interventions, intervention type; study randomization, phase and sample size; analysis type, follow-up period; reported outcomes (median and landmark overall survival (OS), progression-free survival (PFS) and other progression measures, response data, quality of life data); subgroup analyses, safety/toxicity. Accuracy of review and extraction was evaluated on publications in three cancer types: NSCLC, PC, BC compared to human results. Results: Our agentic LLM system generated annotations for 4 review variables for 19,407 publications (6,916 NSCLC, 6,978 PC, 5,513 BC) publications, and 32 extraction variables for 2,424 (1,356 NSCLC, 587 PC, 481 BC) publications. Accuracy for the review variables ranged from 93.8% to 97.2%. For extraction variables, accuracy exceeded 90% for all variables (91.5%-99.1%) with 50% of variables above 95%. Our system completed the annotations in 5.34 hours, compared to an estimated 727.39 hours by trained human researchers, resulting in 99.27% time savings. Conclusions: Our living SLR system can accurately review and extract clinical trial publications with performance comparable to human experts. This level of accuracy highlights our system’s potential to deliver real-time clinical data, empowering oncologists to make more informed treatment decisions, with the hopes of ultimately improving patient outcomes.

Background: The deployment of large language models (LLMs) in mental health therapy presents a compelling yet deeply fraught opportunity to address widespread disparities in access to psychological care. Recent empirical evidence reveals that these AI systems exhibit substantial shortcomings when confronted with complex clinical contexts. Methods: This paper synthesizes key findings from a critical analysis of LLMs operating in therapeutic roles and argues for the urgent establishment of comprehensive risk management frameworks, policy interventions, and ethical protocols governing their use. Results: LLMs tested in simulated therapeutic settings frequently exhibited stigmatizing attitudes toward mental health conditions and responded inappropriately to acute clinical symptoms such as suicidal ideation, psychosis, and delusions. Real-world evaluations reinforce these concerns. Some studies found that therapy and companion bots endorsed unsafe or harmful suggestions in adolescent crisis vignettes, while others reported inadequate chatbot responses to self-harm and sexual assault queries, prompting concern from clinicians, disappointment from patients, and calls for stronger oversight from policymakers. These failures contravene fundamental principles of safe clinical practice, including non-maleficence, therapeutic alliance, and evidence-based care. Moreover, LLMs lack the emotional intelligence, contextual grounding, and ethical accountability that underpin the professional responsibilities of human therapists. Their propensity for sycophantic or non-directive responses, driven by alignment objectives rather than clinical efficacy, further undermines their therapeutic utility. Conclusions: This analysis highlights barriers to the replacement of human therapists with autonomous AI systems. It also calls attention to the regulatory vacuum surrounding LLM-based wellness and therapy applications, many of which are widely accessible and unvetted. Recommendations include professional standards, transparency in training and deployment, robust privacy protections, and clinician oversight. The findings underscore the need to redefine AI as supportive, not substitutive.

The rise of Large Language Models (LLMs) has impacted research in robotics and automation. While progress has been made in integrating LLMs into general robotics tasks, a noticeable void persists in their adoption in more specific domains such as surgery, where critical factors such as reasoning, explainability, and safety are paramount. Achieving autonomy in robotic surgery, which entails the ability to reason and adapt to changes in the environment, remains a significant challenge. In this work, we propose a multi-modal LLM integration in robot-assisted surgery for autonomous blood suction. The reasoning and prioritization are delegated to the higher-level task-planning LLM, and the motion planning and execution are handled by the lower-level deep reinforcement learning model, creating a distributed agency between the two components. As surgical operations are highly dynamic and may encounter unforeseen circumstances, blood clots and active bleeding were introduced to influence decision-making. Results showed that using a multi-modal LLM as a higher-level reasoning unit can account for these surgical complexities to achieve a level of reasoning previously unattainable in robot-assisted surgeries. These findings demonstrate the potential of multi-modal LLMs to significantly enhance contextual understanding and decision-making in robotic-assisted surgeries, marking a step toward autonomous surgical systems.

Autonomous web-based geographical information systems (AWebGIS) aim to perform geospatial operations from natural language input, providing intuitive, intelligent, and hands-free interaction directly within a Web environment. However, most current solutions rely on cloud-based large language models (LLMs), which suffer from centralized processing issues like high operational costs and limited user privacy. This study explores an alternative architectural direction by deploying fine-tuned Small Language Models (SLMs) directly inside the user’s web browser to support AWebGIS. We compare two distinct approaches: (i) a method utilizing a cloud-based LLM (DeepSeek Chat V3.1), and (ii) a method based on three fine-tuned browser-executable SLMs – T5-small, T5-efficient-mini, and T5-efficient-tiny – executed entirely on the user’s device via WebAssembly/WebGPU. The T5-small model (60 M parameters, 155 MB) achieved an Exact Match Accuracy (EMA) of 0.82, Levenshtein Similarity (LS) of 0.96, ROUGE-1, and ROUGE-L scores of 0.95. Even the smallest model, T5-efficient-tiny (16 M parameters, 59 MB), maintained a functional EMA of 0.56. This browser-native, on-device strategy reduces the computational load on cloud servers, eliminates the need for large infrastructures to host LLMs, avoids token-based operational costs, and enhances user privacy. These results demonstrate the feasibility of leveraging highly compressed, browser-executable SLMs for developing cost-efficient, installation-free, and decentralized AWebGIS.

Large language models (LLMs) offer significant potential in enhancing psychiatric practice, from improving diagnostic accuracy to streamlining clinical documentation and therapeutic support. However, existing evaluation resources heavily rely on small clinical interview corpora, social media posts, or synthetic dialogs, which limit their clinical validity and fail to capture the full complexity of diagnostic reasoning. In this work, we introduce PsychiatryBench, a rigorously curated benchmark grounded exclusively in authoritative, expert-validated psychiatric textbooks and casebooks. PsychiatryBench comprises eleven distinct question-answering tasks ranging from diagnostic reasoning and treatment planning to longitudinal follow-up, management planning, clinical approach, sequential case analysis, and multiple-choice/extended matching formats totaling 5,188 expert-annotated items. We evaluate a diverse set of frontier LLMs (including Google Gemini, DeepSeek, Sonnet 4.5, and GPT 5) alongside leading open-source medical models such as MedGemma using both conventional metrics and an "LLM-as-judge" similarity scoring framework. Our results reveal substantial gaps in clinical consistency and safety, particularly in multi-turn follow-up and management tasks, underscoring the need for specialized model tuning and more robust evaluation paradigms. PsychiatryBench offers a modular, extensible platform for benchmarking and improving LLM performance in mental health applications.

The introduction of foundational models, specifically large language models, has promised a health care transformation. However, the field is rapidly evolving toward autonomous agent systems, defined as artificial intelligence (AI) entities that perceive and react to their environment to achieve specific goals-representing a paradigm shift from passive information retrieval to proactive, goal-oriented clinical assistance. Agentic AI systems transcend static knowledge limitations through core capabilities including persistent memory systems that maintain context across patient encounters, knowledge retrieval tools connecting to medical repositories through retrieval-augmented generation techniques, and computer use functionality enabling navigation of clinical software interfaces. Agentic workflows introduce sophisticated coordination mechanisms including hierarchical, collaborative, and sequential patterns demonstrating superior performance compared with single-agent approaches. Multiagent systems can autonomously coordinate entire clinical workflows across the entire radiology life cycle, from preacquisition protocol optimization through initial image analysis, specialized tool deployment, and preliminary report generation. However, successful clinical deployment requires systematic consideration of complexity thresholds, economic sustainability, cybersecurity frameworks, bias mitigation strategies, and appropriate governance structures. Critical challenges include managing the probabilistic nature of underlying models within deterministic clinical workflows, ensuring adequate human supervision, and preventing overcomplication of established processes. A structured four-phase implementation roadmap addresses these considerations through incremental progression from low-risk automation to comprehensive workflow orchestration while maintaining rigorous safety standards. As foundation models advance and interoperability standards mature, agentic AI will reshape radiology practice paradigms. Success depends on resolving stakeholder responsibility questions while orchestrating technological capabilities with clinical accountability, ensuring autonomous systems augment rather than replace professional judgment in pursuit of improved patient outcomes. Keywords: Informatics, Named Entity Recognition, Patient Scheduling/No-Show Prediction, Resource Allocation, Impact of AI on Education, Artificial Intelligence, Large Language Models, Agentic AI, Multi-Agent Systems, Radiology Workflow, Clinical Decision Support, Health Care Automation © RSNA, 2026.

DiagX-DT: diagnostic exclusion reasoning with dialectical thinking for traditional Chinese medicine in large language models

ObjectiveTo evaluate the ability of large language models (LLMs) to perform structured information extraction and guideline-based clinical inferences from radiology and pathology reports in real-world oncology.MethodsWe constructed a Question Answering (Q&A) benchmark dataset using 3650 radiological and 588 pathological reports from 1632 patients. The tasks included direct extraction of genomic and histological findings, as well as clinical reasoning tasks, such as Response Evaluation Criteria in Solid Tumors (RECIST)-based tumor response classification and American Joint Committee on Cancer (AJCC)-based tumor-node-metastasis (TNM) staging. We compared the performance of the Gemma family of open-source LLMs (Gemma 4B, a lightweight 4-billion parameter model, and Gemma 12B, a larger 12-billion parameter model) with and without structured reasoning prompts designed according to clinical guidelines.ResultsThe 12B model achieved high performance in direct extraction tasks from pathology reports, with F1-score ranging from 92.6 to 93.3 across genomic and histological variables. Furthermore, when guided by structured reasoning prompts, it also showed substantial improvements in reasoning tasks, achieving an F1-score of 81.5 (95% CI: 79.8–83.3) for tumor response, 74.3 (95% CI: 70.8–77.8) for T-stage, 87.1 (95% CI: 85.1–89.0) for N-stage, and 90.8 (95% CI: 89.1–92.2) for M-stage. In contrast, the 4B model showed inconsistent performance and was sometimes degraded under reasoning prompts.ConclusionThis study shows that LLMs can perform complex guideline-based clinical reasoning using real-world radiology reports. By combining the RECIST/AJCC criteria with structured prompts, we demonstrated how LLMs can move beyond surface-level extraction to support nuanced inference in oncology, with implications for future clinical applications.