ModelGen: Automating Semiconductor Parameter Extraction with Large Language Model Agents

This study aimed to evaluate the diagnostic accuracy of multimodal large language models in classifying superior labial frenulum attachments from intraoral photographs using expert consensus as the reference standard. Five experts (two periodontists and three orthodontists) established the consensus standard by classifying frenulum attachments in 117 intraoral images as mucosal, gingival, papillary, and papilla penetrating. The same photographs were then presented to three multimodal large language models (ChatGPT 4o, Gemini 2.5 Pro, and Microsoft Copilot GPT-4), and their diagnostic performance was evaluated using accuracy, sensitivity, specificity, and F1 score. Reliability was assessed using Fleiss' and Cohen's Kappa, and diagnostic performances were compared using Cochran's Q test. Human raters demonstrated almost perfect agreement (κ = 0.838, p < 0.001), whereas large language models showed poor inter-model agreement (κ = -0.124, p < 0.001). ChatGPT achieved slight significant agreement with the consensus (κ = 0.114, p = 0.019), although its clinical relevance was negligible. Gemini (κ = 0.099) and Copilot (κ = 0.027) showed no significant agreement (p > 0.05). Copilot yielded the highest overall accuracy (46.2%), followed by Gemini (44.5%) and ChatGPT (35.0%). The performance of the large language models varied across frenulum types. Current multimodal large language models demonstrate inconsistent and clinically insufficient accuracy in the classification of superior labial frenulum attachments from photographs. Domain-specific training is essential before large language models can be considered reliable diagnostic tools in dentistry.

Sentiment analysis is crucial in understanding user opinions, emotions, and attitudes across various domains. Traditional sentiment analysis methods rely primarily on textual data, limiting their ability to capture the full context of human expression, often including multimodal elements such as images, audio, and videos. Existing approaches struggle with ambiguity, sarcasm, and lack of contextual awareness, reducing accuracy and effectiveness. To address these limitations, we propose a novel framework called Sentiment Analysis using Machine Learning with Multimodal Models (SA-ML-MM), which integrates text, images, and audio inputs using large language models (LLMs) enhanced with multimodal capabilities. Our approach leverages ML-based feature extraction and fusion techniques to improve sentiment classification accuracy. The proposed framework is applied in social media analysis, customer feedback interpretation, and emotion recognition. Experimental results demonstrate that SA-ML-MM significantly outperforms traditional text-based models, achieving higher accuracy and robustness in sentiment prediction. By incorporating multiple data modalities, our approach effectively captures nuanced emotional expressions, making sentiment analysis more precise and reliable.

AI-enabled language models (LMs) to large language models (LLMs) and multimodal large language models (MLLMs) in drug discovery and development

SummarySustained engagement in HIV care and adherence to antiretroviral therapy (ART) are essential for achieving the UNAIDS “95-95-95” targets. Despite increased ART access, disengagement from care remains a significant issue, particularly in sub-Saharan Africa. Traditional machine learning (ML) models have shown moderate success in predicting care disengagement, which would enable early intervention. We develop an enhanced large language model (LLM) fine-tuned with electronic medical records (EMRs) to predict people at risk of disengaging from HIV care in Tanzania and to provide interpretative insights into modifiable risk factors.MethodsWe developed a novel AI model by enhancing a pre-trained LLM (LLaMA 3.1, an open-source pre-trained LLM released by Meta) using routinely collected EMRs from Tanzania’s National HIV Care and Treatment Program from January 1, 2018, to June 30, 2023 (4,809,765 records for 261,192 people) to identify people at risk of disengaging from HIV care or developing adverse outcomes. Outcomes included risk of ART non-adherence, non-suppressed viral load, and loss to follow-up. Models were evaluated internally (Kagera region) and externally (Geita region), with performance compared against state-of-art ML models and zero-shot LLMs. Additionally, a team of HIV physicians in Tanzania assessed the LLM’s predictions along with LLM provided justifications for a subset of patient records to evaluate their clinical relevance and reasoning.FindingsThe enhanced LLMs consistently outperformed the supervised ML model and zero-shot LLMs across all outcomes in both internal and external validation datasets. When focusing on the 25% of PLHIV predicted as most likely to lost-to-follow-up (LTFU), the model correctly identified 78% (2,515 of 3,224) of people living with HIV (PLHIV) genuinely at risk in internal validation and 73% (7,105 of 9,733) in external validation. Attention score analysis indicated that the enhanced LLM focused on keywords such as gaps in follow-up care and ART adherence. The human expert evaluation showed alignment between clinician assessments and the LLM’s predictions in 65% of cases, with experts finding the model’s justifications reasonable and clinically relevant in 92.3% of aligned cases.InterpretationIf implemented in HIV clinics, this LLM-based AI model could help allocate resources efficiently and deliver targeted interventions, improving retention in care and advancing the UNAIDS “95-95-95” targets. By functioning like a clinician–analyzing patient summaries, predicting risks, and offering reasoning–the enhanced LLM could be integrated into clinical workflows to complement human expertise, facilitating timely interventions and informed decision-making. If implemented widely, this human-AI collaboration has the potential to improve health outcomes for people living with HIV and reduce onward transmission.FundingThe study was supported by a grant from the US National Institutes of Health (NIH): NIH NIMH 1R01MH125746.

Large language models (LLMs) are increasingly adopted in educational contexts to provide personalized support to students and teachers. The unprecedented capacity of LLM‐based applications to understand and generate natural language can potentially improve instructional effectiveness and learning outcomes, but the integration of LLMs in education technology has renewed concerns over algorithmic bias, which may exacerbate educational inequalities. Building on prior work that mapped the traditional machine learning life cycle, we provide a framework of the LLM life cycle from the initial development of LLMs to customizing pre‐trained models for various applications in educational settings. We explain each step in the LLM life cycle and identify potential sources of bias that may arise in the context of education. We discuss why current measures of bias from traditional machine learning fail to transfer to LLM‐generated text (eg, tutoring conversations) because text encodings are high‐dimensional, there can be multiple correct responses, and tailoring responses may be pedagogically desirable rather than unfair. The proposed framework clarifies the complex nature of bias in LLM applications and provides practical guidance for their evaluation to promote educational equity. Practitioner notes What is already known about this topic The life cycle of traditional machine learning (ML) applications which focus on predicting labels is well understood. Biases are known to enter in traditional ML applications at various points in the life cycle, and methods to measure and mitigate these biases have been developed and tested. Large language models (LLMs) and other forms of generative artificial intelligence (GenAI) are increasingly adopted in education technologies (EdTech), but current evaluation approaches are not specific to the domain of education. What this paper adds A holistic perspective of the LLM life cycle with domain‐specific examples in education to highlight opportunities and challenges for incorporating natural language understanding (NLU) and natural language generation (NLG) into EdTech. Potential sources of bias are identified in each step of the LLM life cycle and discussed in the context of education. A framework for understanding where to expect potential harms of LLMs for students, teachers, and other users of GenAI technology in education, which can guide approaches to bias measurement and mitigation. Implications for practice and/or policy Education practitioners and policymakers should be aware that biases can originate from a multitude of steps in the LLM life cycle, and the life cycle perspective offers them a heuristic for asking technology developers to explain each step to assess the risk of bias. Measuring the biases of systems that use LLMs in education is more complex than with traditional ML, in large part because the evaluation of natural language generation is highly context‐dependent (eg, what counts as good feedback on an assignment varies). EdTech developers can play an important role in collecting and curating datasets for the evaluation and benchmarking of LLM applications moving forward.

Extracting information from unstructured and semi-structured geoscience literature is a crucial step in conducting geological research. The traditional machine learning extraction paradigm requires a substantial amount of high-quality manually annotated data for model training, which is time-consuming, labor-intensive, and not easily transferable to new fields. Recently, large language models (LLMs) (e.g., ChatGPT, GPT-4, and LLaMA), have shown great performance in various natural language processing (NLP) tasks, such as question answering, machine translation, and text generation. A substantial body of work has demonstrated that LLMs possess strong in-context learning (ICL) and even zero-shot learning capabilities to solve downstream tasks without specifically designed supervised fine-tuning. In this paper, we propose utilizing LLMs for geoscience literature information extraction. Specifically, we design a hierarchical PDF parsing pipeline and an automated knowledge extraction process, which can significantly reduce the need for manual data annotation, assisting geoscientists in literature data mining. For the hierarchical PDF parsing pipeline, firstly, a document layout detection model fine-tuned on geoscience literature is employed for layout detection, obtaining layout detection information for the document. Secondly, based on the document layout information, an optical character content parsing model is used for content parsing, obtaining the text structure and plain text corresponding to the content. Finally, the text structure and plain text are combined and reconstructed to ultimately obtain the parsed structured data. For the automated knowledge extraction process, firstly, the parsed long text is segmented into paragraphs to adapt to the input length limit of LLMs. Subsequently, a few-shot prompting method is employed for structured knowledge extraction, encompassing two tasks: attribute value extraction and triplet extraction. In attribute value extraction, prompts are generated automatically by the LLMs based on the subdomain and attribute names, facilitating the location and extraction of values related to subdomain attribute names in the text. For triplet extraction, the LLMs employ a procedural approach to entity extraction, entity type extraction, and relation extraction, following the knowledge graph structure pattern. Finally, the extracted structured knowledge is stored in the form of knowledge graphs, facilitating further analysis and integration of various types of knowledge from the literature. Our proposed approach turns out to be simple, flexible, and highly effective in geoscience literature information extraction. Demonstrations of information extraction in subdomains such as radiolarian fossils and fluvial facies have yielded satisfactory results. The extraction efficiency has significantly improved, and feedback from domain experts indicates a relatively high level of accuracy in the extraction process. The extracted results can be used to construct a foundational knowledge graph for geoscience literature, supporting the comprehensive construction and efficient application of a geoscience knowledge graph.

Advancements in large language model accuracy for answering physical medicine and rehabilitation board review questions.

Background: Accurate identification of cancer progression events from electronic health records (EHRs) can help enable promising oncology applications such as predicting disease trajectory, assessing treatment efficacy, and generating real-world evidence. These use cases require both large-scale and high-quality data but manual abstraction of real-world progression (rwP) is time-intensive, difficult to scale, and inherently challenging given the varied and unstructured ways it can be documented across cancer types. Large language models (LLMs) offer a scalable alternative, but their accuracy relative to expert human abstractors is unclear. We evaluated the ability of LLMs to extract rwP events and dates across 7 cancer types and assessed how using LLM-extracted data impacted real-world progression-free survival (rwPFS) estimates compared to using human-abstracted data. Methods: We applied LLM-based extraction techniques to unstructured EHR text for 7 cancer types from the Flatiron Health Research Database: bladder (N=377), breast (N=1000), colorectal (N=564), hepatocellular (N=217), renal cell (N= 229), non–small cell lung (N=1000), and small cell lung (N=955). Prompt engineering strategies including zero-shot, few-shot, and chain-of-thought were tested to optimize performance. We measured agreement between the LLM and abstractor on the presence of rwP (Y/N) and first rwP date (± 30 days) in the first-line (1L) setting. To contextualize the LLM’s ability to extract rwP relative to an expert human abstractor, we evaluated the difference in F1 scores between both curation approaches using a duplicate human-abstracted reference dataset. We also compared rwPFS calculated from LLM-curated data versus human abstractor-curated data for 1000 patients in each cancer type, indexed to 1L start date. Results: Across all cancer types, agreement between the LLM and abstractor on the presence of at least 1 rwP event ranged from 86%-90% while first rwP date agreement ranged from 80%-92%. The difference in F1 score between the LLM and human abstraction was within 3-8 points across cancer types. A comparison of rwPFS between the LLM and human abstractors showed &amp;lt;1 month difference in median rwPFS and overlapping 95% confidence intervals across all cancer types. Discussion: LLMs extracted rwP with high performance, achieving F1 scores similar to expert human abstraction. Across 7 distinct cancers, agreement with human-abstracted data aligned with published inter-abstractor reliability benchmarks, and rwPFS estimates were nearly identical across curation approaches, demonstrating both the generalizability and validity of the approach. These results highlight the potential of LLMs to extract high-quality clinical endpoints at scale, helping to advance research, enhance applications such as predictive algorithms, and ultimately supporting more personalized and effective cancer care. Citation Format: Aaron B. Cohen, Konstantin Krismer, Kelly Magee, James Gippetti, Aaron Dolor, Tori Williams, Erin Fidyk, Hank Kim, Qianyu Yuan, Melissa Estevez. Using large language models for scalable extraction of real-world progression events across multiple cancer types [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Artificial Intelligence and Machine Learning; 2025 Jul 10-12; Montreal, QC, Canada. Philadelphia (PA): AACR; Clin Cancer Res 2025;31(13_Suppl):Abstract nr B006.

This review explores the frontiers of large language models (LLMs) in psychological applications. Psychology has undergone several theoretical changes, and the current use of artificial intelligence (AI) and machine learning, particularly LLMs, promises to open up new research directions. We aim to provide a detailed exploration of how LLMs are transforming psychological research. We discuss the impact of LLMs across various branches of psychology—including cognitive and behavioral, clinical and counseling, educational and developmental, and social and cultural psychology—highlighting their ability to model patterns, cognition, and behavior similar to those observed in humans. Furthermore, we explore the ability of such models to generate coherent, contextually relevant text, offering innovative tools for literature reviews, hypothesis generation, experimental designs, experimental subjects, and data analysis in psychology. We emphasize the importance of addressing technical and ethical challenges, including data privacy, the ethics of using LLMs in psychological research, and the need for a deeper understanding of these models’ limitations. Researchers should use LLMs responsibly in psychological studies, adhering to ethical standards and considering the potential consequences of deploying these technologies in sensitive areas. Overall, this review provides a comprehensive overview of the current state of LLMs in psychology, exploring the potential benefits and challenges. We hope it can serve as a call to action for researchers to responsibly leverage LLMs’ advantages while addressing the associated risks.

The integration of visual encoders and large language models (LLMs) has driven recent progress in multimodal large language models (MLLMs). However, the scarcity of high-quality instruction-tuning data for vision-language tasks remains a challenge. The current leading paradigm, such as LLaVA, relies on language-only GPT-4 to generate data, which requires pre-annotated image captions and detection bounding boxes, suffering from understanding image details. A practical solution to this problem would be to utilize the available multimodal large language models to generate instruction data for vision-language tasks. However, it's worth noting that the currently accessible MLLMs are not as powerful as their LLM counterparts, as they tend to produce inadequate responses and generate false information. As a solution for addressing the current issue, this paper proposes the Visual Instruction Generation and Correction (VIGC) framework that enables multimodal large language models to generate instruction-tuning data and progressively enhance its quality on-the-fly. Specifically, Visual Instruction Generation (VIG) guides the vision-language model to generate diverse instruction-tuning data. To ensure generation quality, Visual Instruction Correction (VIC) adopts an iterative update mechanism to correct any inaccuracies in data produced by VIG, effectively reducing the risk of hallucination. Leveraging the diverse, high-quality data generated by VIGC, we finetune mainstream models and validate data quality based on various evaluations. Experimental results demonstrate that VIGC not only compensates for the shortcomings of language-only data generation methods, but also effectively enhances the benchmark performance. The models, datasets, and code are available at https://opendatalab.github.io/VIGC

The rise of Large Language Models (LLMs) has impacted research in robotics and automation. While progress has been made in integrating LLMs into general robotics tasks, a noticeable void persists in their adoption in more specific domains such as surgery, where critical factors such as reasoning, explainability, and safety are paramount. Achieving autonomy in robotic surgery, which entails the ability to reason and adapt to changes in the environment, remains a significant challenge. In this work, we propose a multi-modal LLM integration in robot-assisted surgery for autonomous blood suction. The reasoning and prioritization are delegated to the higher-level task-planning LLM, and the motion planning and execution are handled by the lower-level deep reinforcement learning model, creating a distributed agency between the two components. As surgical operations are highly dynamic and may encounter unforeseen circumstances, blood clots and active bleeding were introduced to influence decision-making. Results showed that using a multi-modal LLM as a higher-level reasoning unit can account for these surgical complexities to achieve a level of reasoning previously unattainable in robot-assisted surgeries. These findings demonstrate the potential of multi-modal LLMs to significantly enhance contextual understanding and decision-making in robotic-assisted surgeries, marking a step toward autonomous surgical systems.

We present a systematic evaluation of modern multimodal large language models (LLMs) for the classification of mean-motion and secular resonances from images of resonant arguments. Four benchmark datasets (RB-TEST, RB-PILOT, RB-SMALL, RB-FULL) were constructed to cover clear, ambiguous, and transient cases, with both binary and three-class outputs. Using standardized prompts (a full prompt for large models and a simplified variant for small models that cannot process complex instructions), we tested flagship commercial models, large open-source models, and small locally runnable models. Commercial LLMs reach [Formula: see text] on simple cases and up to [Formula: see text] on the three-class RB-SMALL dataset, while the best open-source models also reach [Formula: see text] on unambiguous cases and [Formula: see text] on the complex ones. On the full binary benchmark, open-source models approach commercial performance ([Formula: see text]-[Formula: see text]). Most errors occur in transient and resonance-sticking regimes. The results show that LLMs can perform resonance classification at levels comparable to those of classical or machine-learning methods without training or fine-tuning, and that even small open-source models achieve practically useful accuracy. The released benchmarks establish a reproducible standard for evaluating LLMs on dynamical astronomy tasks.

BackgroundDiagnosing oral lesions from benign conditions to oral cancer remains challenging due to overlapping visual features and reliance on histopathology. Large language models (LLMs) can integrate textual and visual cues, but their diagnostic accuracy and clinical utility in real decision-making contexts remain uncertain. To systematically evaluate the diagnostic performance, clinical usefulness, and limitations of LLMs in identifying oral lesions.MethodsPubMed, CINAHL, Embase, Web of Science, and Google Scholar were searched to 20 July 2025. Eligible studies applied LLMs (e.g., ChatGPT, Gemini, DeepSeek, Copilot, Claude) for diagnosis or differential diagnosis of oral lesions using text, images, or multimodal inputs. Outcomes included diagnostic accuracy, agreement metrics, and qualitative assessments of explanation quality and clinical applicability. Risk of bias was assessed using an adapted QUADAS-2. Narrative synthesis was performed due to heterogeneity.ResultsSeventeen studies (>1,200 cases) were included. Diagnostic accuracy ranged from 25%–96%, varying by model version, input modality, and lesion complexity. Multimodal inputs consistently improved performance, with Cohen's κ up to 0.85–0.90. Advanced models (GPT-4o, DeepSeek-R1, o1-preview) outperformed earlier versions and approached expert performance in some tasks, although specialists generally retained superior Top-1 accuracy. Clinical utility was highest when LLMs were used to structure differential reasoning, highlight red-flag features, and support communication, but limited in tasks requiring fine morphological interpretation or severity grading. Overall risk of bias was low to moderate.ConclusionsLLMs demonstrate variable diagnostic performance and context-dependent supportive utility as adjunctive tools in oral lesion assessment, particularly in multimodal settings. They should complement, rather than replace, expert clinical judgment. Future research should prioritize real-world workflow evaluation, standardized prompting strategies, and prospective clinical validation.Systematic Review Registrationhttps://www.crd.york.ac.uk/PROSPERO/view/CRD420251090315, identifier CRD420251090315.

Large language models (LLMs) have opened up new possibilities for intelligent agents, endowing them with human-like thinking and cognitive abilities. In this work, we delve into the potential of large language models (LLMs) in autonomous driving (AD). We introduce DriveMLM, an LLM-based AD framework that can perform close-loop autonomous driving in realistic simulators. To this end, (1) we bridge the gap between the language decisions and the vehicle control commands by standardizing the decision states according to the off-the-shelf motion planning module. (2) We employ a multimodal LLM (MLLM) to model the behavior planning module of a module AD system, which uses driving rules, user commands, and inputs from various sensors (e.g., camera, LiDAR) as input and makes driving decisions and provide explanations. This model can plug-and-play in existing AD systems such as Autopilot and Apollo for close-loop driving. (3) We design an effective data engine to collect a dataset that includes decision state and corresponding explanation annotation for model training and evaluation. We conduct extensive experiments and show that replacing the decision-making modules of the Autopilot and Apollo with DriveMLM resulted in significant improvements of 3.2 and 4.7 points on the CARLA Town05 Long, respectively, demonstrating the effectiveness of our model. We hope this work can serve as a baseline for autonomous driving with LLMs.

The generation of corner cases has become increasingly crucial for efficiently testing autonomous vehicles prior to road deployment. However, existing methods struggle to accommodate diverse testing requirements and often lack the ability to generalize to unseen situations, thereby reducing the convenience and usability of the generated scenarios. A method that facilitates easily controllable scenario generation for efficient autonomous vehicles (AV) testing with realistic and challenging situations is greatly needed. To address this, OmniTester is proposed as a multimodal Large Language Model (LLM) based framework that fully leverages the extensive world knowledge and reasoning capabilities of LLMs. OmniTester is designed to generate realistic and diverse scenarios within a simulation environment, offering a robust solution for testing and evaluating AVs. In addition to prompt engineering, OmniTester employs tools from Simulation of Urban Mobility to simplify the complexity of codes generated by LLMs. It further incorporates Retrieval-Augmented Generation and a self-improvement mechanism to enhance the LLM's understanding of scenarios, thereby increasing its ability to produce more realistic scenes. Experiment results demonstrated the controllability and realism of the proposed approaches in generating three types of challenging and complex scenarios. Additionally, OmniTester effectively reconstructs novel scenarios described in crash reports, driven by the generalization capability of LLMs.