ChatMol: interactive molecular discovery with natural language.

Simulation modeling provides powerful decision support capabilities for production planning in manufacturing systems, yet widespread adoption remains limited by significant barriers in model development and maintenance. Traditional simulation approaches require substantial expertise, time investment, and financial resources, creating accessibility challenges particularly for small and medium enterprises. Current research on automatic simulation generation overlooks the significant potential that generative AI can offer in bridging the gap between natural language expression and executable model creation. This paper presents a novel methodology integrating large language models (LLMs) with automatic simulation model generation (ASMG) to enable direct generation of executable simulation models from natural language conversations. Unlike existing ASMG approaches that require formal specifications or simulation knowledge, our framework reduces programming requirements by allowing users to create functional models through conversational interaction. Our approach leverages natural language processing capabilities to transform user requests into executable simulation models without requiring specialized programming knowledge, enhancing human–machine interaction in production planning environments. The framework implements a four-step process: user intent expression through natural language interfaces, knowledge extraction using structured templates, simulation model construction via object-oriented data-driven components, and simulation execution with comprehensive result analysis for resource utilization, bottleneck identification, and production scheduling optimization. Performance testing demonstrates successful model generation across 18 supported action types with response times ranging from 8 s for simple operations to 5 min for complete manufacturing system creation. By enabling direct conversion of natural language requests into functional simulation models, this LLM-enhanced ASMG approach represents a step toward expanding access to simulation technology for non-expert users while maintaining the analytical power needed for complex manufacturing decision support and production control.

Generative large language models (LLMs) are a subset of transformers-based neural network architecture models. LLMs have successfully leveraged a combination of an increased number of parameters, improvements in computational efficiency, and large pre-training datasets to perform a wide spectrum of natural language processing (NLP) tasks. Using a few examples (few-shot) or no examples (zero-shot) for prompt-tuning has enabled LLMs to achieve state-of-the-art performance in a broad range of NLP applications. This article by the American Medical Informatics Association (AMIA) NLP Working Group characterizes the opportunities, challenges, and best practices for our community to leverage and advance the integration of LLMs in downstream NLP applications effectively. This can be accomplished through a variety of approaches, including augmented prompting, instruction prompt tuning, and reinforcement learning from human feedback (RLHF). Our focus is on making LLMs accessible to the broader biomedical informatics community, including clinicians and researchers who may be unfamiliar with NLP. Additionally, NLP practitioners may gain insight from the described best practices. We focus on 3 broad categories of NLP tasks, namely natural language understanding, natural language inferencing, and natural language generation. We review the emerging trends in prompt tuning, instruction fine-tuning, and evaluation metrics used for LLMs while drawing attention to several issues that impact biomedical NLP applications, including falsehoods in generated text (confabulation/hallucinations), toxicity, and dataset contamination leading to overfitting. We also review potential approaches to address some of these current challenges in LLMs, such as chain of thought prompting, and the phenomena of emergent capabilities observed in LLMs that can be leveraged to address complex NLP challenge in biomedical applications.

Designing preference elicitation (PE) methodologies that can quickly ascertain a user’s top item preferences in a cold-start setting is a key challenge for building effective and personalized conversational recommendation (ConvRec) systems. While large language models (LLMs) enable fully natural language (NL) PE dialogues, we hypothesize that monolithic LLM NL-PE approaches lack the multi-turn, decision-theoretic reasoning required to effectively balance the exploration and exploitation of user preferences towards an arbitrary item set. In contrast, traditional Bayesian optimization PE methods define theoretically optimal PE strategies, but cannot generate arbitrary NL queries or reason over content in NL item descriptions – requiring users to express preferences via ratings or comparisons of unfamiliar items. To overcome the limitations of both approaches, we formulate NL-PE in a Bayesian Optimization (BO) framework that seeks to actively elicit NL feedback to identify the best recommendation. Key challenges in generalizing BO to deal with natural language feedback include determining: (a) how to leverage LLMs to model the likelihood of NL preference feedback as a function of item utilities, and (b) how to design an acquisition function for NL BO that can elicit preferences in the infinite space of language. We demonstrate our framework in a novel NL-PE algorithm, PEBOL, which uses: 1) Natural Language Inference (NLI) between user preference utterances and NL item descriptions to maintain Bayesian preference beliefs, and 2) BO strategies such as Thompson Sampling (TS) and Upper Confidence Bound (UCB) to guide LLM query generation. We numerically evaluate our methods in controlled simulations, finding that after 10 turns of dialogue, PEBOL can achieve an MRR@10 of up to 0.27 compared to the best monolithic LLM baseline’s MRR@10 of 0.17, despite relying on earlier and smaller LLMs.1

Recommendation models typically follow a discriminative paradigm, predicting whether items should be retrieved. While effective, the expressive capabilities of these recommender systems are limited. Users can only passively browse the recommended items rather than actively express their needs and engage in an interactive experience. With recent advances in generative models such as large language models, a paradigm shift is happening in the study of recommender systems. Researchers propose building generative recommendation models either by aligning pre-trained generative models with user behaviors or designing recommendation models within a generative framework. These models enable the systems to receive and deliver more human-like content, such as natural language, images, and beyond. In this tutorial, we first provide an overview of the latest progress in generative recommendation models, covering approaches based on large language models, semantic IDs, diffusion models, and more. We then make an in-depth discussion on the challenges, open questions, and potential future directions in developing generative recommendation models.

Existing methods, including large language models (LLMs), excel at capturing semantic correlations between utterances, but often struggle to accurately distinguish specific causal relationships. This limitation poses a significant challenge for reasoning-intensive tasks in affective computing, where precise identification of emotional triggers and their effects is crucial. Our preliminary work demonstrated the potential of introducing i.i.d. noise terms within Structural Causal Models (SCMs) for the Emotion-Cause Pair Extraction (ECPE) task. However, this approach relied on end-to-end learning of high-dimensional latent representations, which hindered both scalability to LLMs and model interpretability. To address these issues, we conceptualize i.i.d. noise terms as token-level implicit causes—natural language expressions that reflect a speaker's underlying emotions, intentions, or situational context. Building on this insight, we introduce ICE (Implicit-Cause-Enhanced), an instruction-based framework that leverages implicit causes to enhance causal reasoning in LLMs. First, we design prompts that heuristically guide LLMs to generate implicit causes, which are then iteratively refined via an external evaluation mechanism. Second, by incorporating these implicit causes as intermediate reasoning steps, ICE improves the accuracy of emotion-cause pair prediction. Moreover, we distill the rationales produced by ICE into lightweight generative models, demonstrating that even small models can benefit from implicit-cause-driven reasoning. Extensive experiments in both instruction-based and distillation-based settings confirm the effectiveness, robustness, and interpretability of our approach.

Large language models (LLMs) are increasingly adopted in educational contexts to provide personalized support to students and teachers. The unprecedented capacity of LLM‐based applications to understand and generate natural language can potentially improve instructional effectiveness and learning outcomes, but the integration of LLMs in education technology has renewed concerns over algorithmic bias, which may exacerbate educational inequalities. Building on prior work that mapped the traditional machine learning life cycle, we provide a framework of the LLM life cycle from the initial development of LLMs to customizing pre‐trained models for various applications in educational settings. We explain each step in the LLM life cycle and identify potential sources of bias that may arise in the context of education. We discuss why current measures of bias from traditional machine learning fail to transfer to LLM‐generated text (eg, tutoring conversations) because text encodings are high‐dimensional, there can be multiple correct responses, and tailoring responses may be pedagogically desirable rather than unfair. The proposed framework clarifies the complex nature of bias in LLM applications and provides practical guidance for their evaluation to promote educational equity. Practitioner notes What is already known about this topic The life cycle of traditional machine learning (ML) applications which focus on predicting labels is well understood. Biases are known to enter in traditional ML applications at various points in the life cycle, and methods to measure and mitigate these biases have been developed and tested. Large language models (LLMs) and other forms of generative artificial intelligence (GenAI) are increasingly adopted in education technologies (EdTech), but current evaluation approaches are not specific to the domain of education. What this paper adds A holistic perspective of the LLM life cycle with domain‐specific examples in education to highlight opportunities and challenges for incorporating natural language understanding (NLU) and natural language generation (NLG) into EdTech. Potential sources of bias are identified in each step of the LLM life cycle and discussed in the context of education. A framework for understanding where to expect potential harms of LLMs for students, teachers, and other users of GenAI technology in education, which can guide approaches to bias measurement and mitigation. Implications for practice and/or policy Education practitioners and policymakers should be aware that biases can originate from a multitude of steps in the LLM life cycle, and the life cycle perspective offers them a heuristic for asking technology developers to explain each step to assess the risk of bias. Measuring the biases of systems that use LLMs in education is more complex than with traditional ML, in large part because the evaluation of natural language generation is highly context‐dependent (eg, what counts as good feedback on an assignment varies). EdTech developers can play an important role in collecting and curating datasets for the evaluation and benchmarking of LLM applications moving forward.

This paper is devoted to present a new approach to natural language understanding which is called here goal-oriented parsing. The interaction in natural language with artificial systems (robots, data base systems, program generators, question-answering systems, etc.) does not require in most cases of actual interest a full (human like) comprehension of natural language in all its details and nuances. A partial understanding is often enough, wich extracts from the natural language expressions the only significant information which is necessary to construct a correct formal input for the target system. In such a model of comprehension the same meaning is assigned to several different natural language expressions, thus defining a many-to-one mapping between natural language sentences and corresponding formal representations. We argue that a bounded scope, restricted, goal-oriented understanding of natural language may greatly increase the efficency of representation models and parsing algorithms, thus allowing the construction of effective systems. This claim is supported by the design and implementation of a natural language interface to a relational data base called NLI and developed at the Milan Polytechnic Artificial Intelligence Project. In the paper the architecture of the system, the linguistic models, and the parsing algorithms are presented and illustrated through selected examples. Promising directions for future research are outlined as well.

Interacting with Large Language Models (LLMs) via declarative queries is increasingly popular for tasks like question answering and data extraction, thanks to their ability to process vast unstructured data. However, LLMs often struggle with answering complex factual questions, exhibiting low precision and recall in the returned data. This challenge highlights that executing queries on LLMs remains a largely unexplored domain, where traditional data processing assumptions often fall short. Conventional query optimization, typically cost-driven, overlooks LLM-specific quality challenges such as contextual understanding. Just as new physical operators are designed to address the unique characteristics of LLMs, optimization must consider these quality challenges. Our results highlight that adhering strictly to conventional query optimization principles fails to generate the best plans in terms of result quality. To tackle this challenge, we present a novel approach to enhance SQL results by applying query optimization techniques specifically adapted for LLMs. We introduce a database system, GALOIS, that sits between the query and the LLM, effectively using the latter as a storage layer. We design alternative physical operators tailored for LLM-based query execution and adapt traditional optimization strategies to this novel context. For example, while pushing down operators in the query plan reduces execution cost (fewer calls to the model), it might complicate the call to the LLM and deteriorate result quality. Additionally, these models lack a traditional catalog for optimization, leading us to develop methods to dynamically gather such metadata during query execution. Our solution is compatible with any LLM and balances the trade-off between query result quality and execution cost. Experiments show up to 144% quality improvement over questions in Natural Language and 29% over direct SQL execution, highlighting the advantages of integrating database solutions with LLMs.

Proving mathematical theorems using computer-verifiable Formal Languages (FL) like Lean significantly impacts mathematical reasoning.One approach to formal theorem proving involves generating complete proofs using Large Language Models (LLMs) based on Natural Language (NL) proofs.However, due to the scarcity of aligned NL and FL theorem-proving data, most modern LLMs exhibit suboptimal performance.This scarcity results in a paucity of methodologies for training LLMs and techniques to fully utilize their capabilities in composing formal proofs.To address these challenges, this paper proposes TheoremLlama, an end-to-end framework that trains a general-purpose LLM to be a Lean4 expert.TheoremLlama includes NL-FL dataset generation and bootstrapping method to obtain aligned dataset, curriculum learning and block training techniques to train the model, and iterative proof writing method to write Lean4 proofs that work together synergistically.Using the dataset generation method in TheoremLlama, we provide Open Bootstrapped Theorems (OBT), an NL-FL aligned and bootstrapped dataset.Our novel NL-FL bootstrapping method, where NL proofs are integrated into Lean4 code for training datasets, leverages the NL reasoning ability of LLMs for formal reasoning.The TheoremLlama framework achieves cumulative accuracies of 36.48% and 33.61% on MiniF2F-Valid and Test datasets respectively, surpassing the GPT-4 baseline of 22.95% and 25.41%.Our code, model checkpoints, and the generated dataset is published in GitHub

Abstract. Large language models (LLMs) have revolutionized the field of natural language processing (NLP), demonstrating remarkable capabilities in understanding, generating, and manipulating human language. This comprehensive review explores the development, applications, optimizations, and challenges of LLMs. This paper begin by tracing the evolution of these models and their foundational architectures, such as the Transformer, GPT, and BERT. We then delve into the applications of LLMs in natural language understanding tasks, including sentiment analysis, named entity recognition, question answering, and text summarization, highlighting real-world use cases. Next, we examine the role of LLMs in natural language generation, covering areas such as content creation, language translation, personalized recommendations, and automated responses. We further discuss LLM applications in other NLP tasks like text style transfer, text correction, and language model pre-training. Subsequently, we explore techniques for optimizing and improving LLMs, including model compression, explainability, robustness, and security. Finally, we address the challenges posed by the significant computational requirements, sample inefficiency, and ethical considerations surrounding LLMs. We conclude by discussing potential future research directions, such as efficient architectures, few-shot learning, bias mitigation, and privacy-preserving techniques, which will shape the ongoing development and responsible deployment of LLMs in NLP.

Mathematical Problem Solving in Arabic: Assessing Large Language Models

Introduction Game theory has long served as a foundational tool in cybersecurity to test, predict, and design strategic interactions between attackers and defenders. The recent advent of Large Language Models (LLMs) offers new tools and challenges for the security of computer systems. In this work, we investigate whether classical game-theoretic frameworks can effectively capture the behaviors of LLM-driven actors and bots. Methods Using a reproducible framework for game-theoretic LLM agents, we investigate two canonical scenarios—the one-shot zero-sum game and the dynamic Prisoner's Dilemma—and we test whether LLMs converge to expected outcomes or exhibit deviations due to embedded biases. We experiments on four state-of-the-art LLMs and five natural languages (English, French, Arabic, Vietnamese, and Mandarin Chinese) to assess linguistic sensitivity. Results For both games, we observe that the final payoffs are influenced by agents characteristics such as personality traits or knowledge of repeated rounds. We also uncover an unexpected sensitivity of the final payoffs to the choice of languages, which should warn against indiscriminate application of LLMs in cybersecurity applications and call for in-depth studies, as LLMs may behave differently when deployed in different countries. We also employ quantitative metrics to evaluate the internal consistency and cross-language stability of LLM agents. Discussion In addition to uncovering unexpected behaviors requiring attention by scholars and practitioners, our work can help guide the selection of the most stable LLMs and optimizing models for secure applications.

DialogueLLM: Context and emotion knowledge-tuned large language models for emotion recognition in conversations.

Evaluating gpt-4 for zero-shot classification of bleeding and clotting events: Can large language models serve as second reviewers?

This paper addresses the challenge of automating process‐centric quality assurance (QA) in safety‐critical domains, where compliance with regulations is crucial. Currently, QA engineers manually check compliance using tedious methods like browsing engineering artifacts and ad‐hoc scripts. Automated support could improve efficiency, but it requires constraints to be written in structured, executable forms (e.g., in the Object Constraint Language, OCL), whereas engineers prefer natural language. To bridge this gap, we propose the use of large language models (LLMs) to generate OCL from natural language, enhanced by schema‐based prompting and domain‐specific language (DSL)–based repairs. Unlike prior work focused on UML models, this work applies OCL to software process QA. Evaluating six LLMs, we find o1‐mini and Codestral perform best, with our automatic repairs ensuring constraint executability for 22%–44% of an LLM's generated OCL constraints that would otherwise remain nonexecutable due to errors.