Natural language-driven production planning: integrating large language models with automatic simulation model generation in manufacturing systems

Model-Driven Performance Engineering enables the automatic generation of simulation models based on business process models and monitored process instance data. When we applied an initial version of our tooling to a number of real world processes, we experienced that we need to support the mapping of monitored process instance data into simulation models under consideration of cases where confidence in these data is low, for instance due to a high variance in monitored resource demands, or a low number of executed process instances. The current paper proposes an architecture which utilizes a decision tree for the intelligent mapping of monitored process instance data into simulation models and, as a by-product, which ranks uncertainties within the imported data.

Natural language is poised to become a key medium for human-machine interactions in the era of large language models. In the field of biochemistry, tasks such as property prediction and molecule mining are critically important yet technically challenging. Bridging molecular expressions in natural language and chemical language can significantly enhance the interpretability and ease of these tasks. Moreover, it can integrate chemical knowledge from various sources, leading to a deeper understanding of molecules. Recognizing these advantages, we introduce the concept of conversational molecular design, a novel task that utilizes natural language to describe and edit target molecules. To better accomplish this task, we develop ChatMol, a knowledgeable and versatile generative pretrained model. This model is enhanced by incorporating experimental property information, molecular spatial knowledge, and the associations between natural and chemical languages. Several typical solutions including large language models (e.g. ChatGPT) are evaluated, proving the challenge of conversational molecular design and the effectiveness of our knowledge enhancement approach. Case observations and analysis offer insights and directions for further exploration of natural-language interaction in molecular discovery. Codes and data are provided in https://github.com/Ellenzzn/ChatMol/tree/main.

In recent years, the analysis and evaluation of manufacturing systems’ behaviour and their performance became essential. Discrete event simulation (DES) as a digital enterprise technology is an effective tool both in production related decision support processes and in structure or performance analysis of manufacturing systems. Building a DES model of a manufacturing system is a difficult task and requires special competence. Reducing the efforts spent on draft simulation is the aim of an ongoing research which is presented in the paper. The developed framework simplifies and accelerates the process of model building. A production oriented implementation of the ANSI/ISA-95 standard was used by the proposed modelling methodology to define a generic data structure supporting the creation of models without specific knowledge related to the simulation software applied. The data structure enables the development and application of proprietary simulation engines tailored for specific problems. On the other hand, numerous objective tests containing simulation model validation methods are also announced through a well-described example problem. By this method, several validating key performance indicators (KPIs) were tested, and it was indirectly determined whether the simulation model adequately represents the behaviour of the real system. The automatic simulation model generation in our approach enables using different simulation tools and presented through the examples of both commercial and self-developed software.

Simulation languages and the GUIs supporting them may be excellent tools for creating simulation codes, but are not necessarily the best tools to use for creating descriptions of systems, i. e., for modeling. In 2006, OMG published the initial standard specification (OMG 2006) for SysML (Systems Modeling Language), an extension of UML (OMG 2007) designed specifically to support systems engineering. SysML shows great promise for creating object-oriented models of systems that incorporate not only software, but also people, material, and other physical resources, expressing both structure and behavior for such systems. In this paper, we explore the use of SysML both to model a system to be simulated and to support the automatic generation of simulation models.

The automatic generation of simulation models has been a recurring research topic for several years. In manufacturing industries, it is currently also becoming a topic of high practical relevance. A well-known challenge in most model generation approaches is the correct reproduction of the dynamic behaviour of model elements, for example, buffering or control strategies. This problem is especially relevant in simulation-based manufacturing control. In such scenarios, simulation models need to reflect the current state and behaviour of the real system in a highly accurate way, otherwise its suggested control decisions may be inaccurate or even dangerous towards production goals. This paper introduces a novel methodology for approximating dynamic behaviour using artificial neural networks, rather than trying to determine exact representations. We suggest using neural networks in conjunction with traditional material flow simulation systems whenever a certain decision cannot be made ex ante in the model generation process due to insufficient knowledge about the behaviour of the real system. In such cases the decision is delegated to the neural network, which is connected to the simulation system at runtime. Training of the neural network is performed by observation of the real systems decision and based on the evaluation of data that can be gained through production data acquisition. Our approach has certain advantages compared to other approaches and is especially well suited in the context of on-line simulation and simulation-based operational decision support. We demonstrate the applicability of our methodology using a case study and report on performance results.

This paper introduces a novel method utilizing speech-based digital assistants and large language models (LLMs) to streamline the creation of simulation models for Job Shop Scheduling Problems (JSSP). The system simplifies the process by allowing natural language interactions for ontology-based model generation. The study evaluates the performance of various LLMs in ontology-based simulation modeling by benchmarking their ability to extract and assign semantical entities and relations. We found that ChatGPT-4-Turbo is able to correctly identify all model elements given in descriptions of the production scenarios we tested, while less resource-intensive and open source models like Mixtral-8x7b and Zephyr-beta perform well in a less complex scenario. The findings demonstrate the potential of integrating LLMs and natural language processing in simulation modeling, significantly enhancing efficiency and reducing the need for manual modeling.

Generative large language models (LLMs) are a subset of transformers-based neural network architecture models. LLMs have successfully leveraged a combination of an increased number of parameters, improvements in computational efficiency, and large pre-training datasets to perform a wide spectrum of natural language processing (NLP) tasks. Using a few examples (few-shot) or no examples (zero-shot) for prompt-tuning has enabled LLMs to achieve state-of-the-art performance in a broad range of NLP applications. This article by the American Medical Informatics Association (AMIA) NLP Working Group characterizes the opportunities, challenges, and best practices for our community to leverage and advance the integration of LLMs in downstream NLP applications effectively. This can be accomplished through a variety of approaches, including augmented prompting, instruction prompt tuning, and reinforcement learning from human feedback (RLHF). Our focus is on making LLMs accessible to the broader biomedical informatics community, including clinicians and researchers who may be unfamiliar with NLP. Additionally, NLP practitioners may gain insight from the described best practices. We focus on 3 broad categories of NLP tasks, namely natural language understanding, natural language inferencing, and natural language generation. We review the emerging trends in prompt tuning, instruction fine-tuning, and evaluation metrics used for LLMs while drawing attention to several issues that impact biomedical NLP applications, including falsehoods in generated text (confabulation/hallucinations), toxicity, and dataset contamination leading to overfitting. We also review potential approaches to address some of these current challenges in LLMs, such as chain of thought prompting, and the phenomena of emergent capabilities observed in LLMs that can be leveraged to address complex NLP challenge in biomedical applications.

The DONS system, Designer Of Network Schedules, supports the development of timetables by generating regular interval timetables for the Dutch railway network; it is used to plan improved or new train services and to locate bottlenecks in the railway infrastructure. A new simulation module enables the planners to study the networkwide dynamic properties of DONS timetables. The direct link between DONS and this simulation module, enhanced by the re-use of previously defined DONS concepts and the automatic generation of simulation models from existing data, facilitates an integrated approach towards infrastructure and railway services development, focused on quality improvement.

The paper is oriented to application of locally used formal theories as tools for analysis and modeling of anticipatory systems, and especially of nesting anticipation, which takes place e.g. in anticipation during a design of a system that is expected to be anticipatory one. The specialization of theories enables to “tailor” them near to the studied systems and the nesting of theories enables formulating exact theories of systems, the elements of which carry their proper theories. In certain programming languages such theories make easy computer model construction and especially automatic generation of simulation models.

The design of production and logistic systems is a process of managing both technical and organisational variants in order to identify the best solution for a given system. This paper discusses design issues of production systems that are applied to an internal logistic system in the automotive industry. As far as the production systems design (PSD) is concerned, three basic classes of software tools usually pertain: computer-aided design (CAD), process simulation, and information systems. However, these software tools have been used with low levels of integration. Vik et al. (2010b, 2010c) proposed integrating these software resources in production systems and developed an advanced tool called integrated design of systems (IDS). The proposed IDS tool involves a wide set of functions for the most common tasks of PSD, from conceptualisation to implementation, including systems analysis (P-Q, cluster, and material flow analysis), automatic generation of simulation models, generation of alternatives for the layout of facilities and factories, material flows display, transportation system design, and iterative buffer size specification. The IDS approach takes advantage of simulation, CAD systems, and their integration. This paper will demonstrate the concept and functionalities of the proposed tool in a real industrial case study.

The transition from cyber-physical-system-based (CPS-based) Industry 4.0 to cyber-physical-social-system-based (CPSS-based) Industry 5.0 brings new requirements and opportunities to current sensing approaches, especially in light of recent progress in large language models (LLMs) and retrieval augmented generation (RAG). Therefore, the advancement of parallel intelligence powered crowdsensing intelligence (CSI) is witnessed, which is currently advancing toward linguistic intelligence. In this article, we propose a novel sensing paradigm, namely conversational crowdsensing, for Industry 5.0 (especially for social manufacturing). It can alleviate workload and professional requirements of individuals and promote the organization and operation of diverse workforce, thereby facilitating faster response and wider popularization of crowdsensing systems. Specifically, we design the architecture of conversational crowdsensing to effectively organize three types of participants (biological, robotic, and digital) from diverse communities. Through three levels of effective conversation (i.e., interhuman, human–AI, and inter-AI), complex interactions and service functionalities of different workers can be achieved to accomplish various tasks across three sensing phases (i.e., requesting, scheduling, and executing). Moreover, we explore the foundational technologies for realizing conversational crowdsensing, encompassing LLM-based multiagent systems, scenarios engineering and conversational human–AI cooperation. Finally, we present potential applications of conversational crowdsensing and discuss its implications. We envision that conversations in natural language will become the primary communication channel during crowdsensing process, enabling richer information exchange and cooperative problem-solving among humans, robots, and AI.

Designing preference elicitation (PE) methodologies that can quickly ascertain a user’s top item preferences in a cold-start setting is a key challenge for building effective and personalized conversational recommendation (ConvRec) systems. While large language models (LLMs) enable fully natural language (NL) PE dialogues, we hypothesize that monolithic LLM NL-PE approaches lack the multi-turn, decision-theoretic reasoning required to effectively balance the exploration and exploitation of user preferences towards an arbitrary item set. In contrast, traditional Bayesian optimization PE methods define theoretically optimal PE strategies, but cannot generate arbitrary NL queries or reason over content in NL item descriptions – requiring users to express preferences via ratings or comparisons of unfamiliar items. To overcome the limitations of both approaches, we formulate NL-PE in a Bayesian Optimization (BO) framework that seeks to actively elicit NL feedback to identify the best recommendation. Key challenges in generalizing BO to deal with natural language feedback include determining: (a) how to leverage LLMs to model the likelihood of NL preference feedback as a function of item utilities, and (b) how to design an acquisition function for NL BO that can elicit preferences in the infinite space of language. We demonstrate our framework in a novel NL-PE algorithm, PEBOL, which uses: 1) Natural Language Inference (NLI) between user preference utterances and NL item descriptions to maintain Bayesian preference beliefs, and 2) BO strategies such as Thompson Sampling (TS) and Upper Confidence Bound (UCB) to guide LLM query generation. We numerically evaluate our methods in controlled simulations, finding that after 10 turns of dialogue, PEBOL can achieve an MRR@10 of up to 0.27 compared to the best monolithic LLM baseline’s MRR@10 of 0.17, despite relying on earlier and smaller LLMs.1

Large language models (LLMs) are increasingly adopted in educational contexts to provide personalized support to students and teachers. The unprecedented capacity of LLM‐based applications to understand and generate natural language can potentially improve instructional effectiveness and learning outcomes, but the integration of LLMs in education technology has renewed concerns over algorithmic bias, which may exacerbate educational inequalities. Building on prior work that mapped the traditional machine learning life cycle, we provide a framework of the LLM life cycle from the initial development of LLMs to customizing pre‐trained models for various applications in educational settings. We explain each step in the LLM life cycle and identify potential sources of bias that may arise in the context of education. We discuss why current measures of bias from traditional machine learning fail to transfer to LLM‐generated text (eg, tutoring conversations) because text encodings are high‐dimensional, there can be multiple correct responses, and tailoring responses may be pedagogically desirable rather than unfair. The proposed framework clarifies the complex nature of bias in LLM applications and provides practical guidance for their evaluation to promote educational equity. Practitioner notes What is already known about this topic The life cycle of traditional machine learning (ML) applications which focus on predicting labels is well understood. Biases are known to enter in traditional ML applications at various points in the life cycle, and methods to measure and mitigate these biases have been developed and tested. Large language models (LLMs) and other forms of generative artificial intelligence (GenAI) are increasingly adopted in education technologies (EdTech), but current evaluation approaches are not specific to the domain of education. What this paper adds A holistic perspective of the LLM life cycle with domain‐specific examples in education to highlight opportunities and challenges for incorporating natural language understanding (NLU) and natural language generation (NLG) into EdTech. Potential sources of bias are identified in each step of the LLM life cycle and discussed in the context of education. A framework for understanding where to expect potential harms of LLMs for students, teachers, and other users of GenAI technology in education, which can guide approaches to bias measurement and mitigation. Implications for practice and/or policy Education practitioners and policymakers should be aware that biases can originate from a multitude of steps in the LLM life cycle, and the life cycle perspective offers them a heuristic for asking technology developers to explain each step to assess the risk of bias. Measuring the biases of systems that use LLMs in education is more complex than with traditional ML, in large part because the evaluation of natural language generation is highly context‐dependent (eg, what counts as good feedback on an assignment varies). EdTech developers can play an important role in collecting and curating datasets for the evaluation and benchmarking of LLM applications moving forward.

Existing methods, including large language models (LLMs), excel at capturing semantic correlations between utterances, but often struggle to accurately distinguish specific causal relationships. This limitation poses a significant challenge for reasoning-intensive tasks in affective computing, where precise identification of emotional triggers and their effects is crucial. Our preliminary work demonstrated the potential of introducing i.i.d. noise terms within Structural Causal Models (SCMs) for the Emotion-Cause Pair Extraction (ECPE) task. However, this approach relied on end-to-end learning of high-dimensional latent representations, which hindered both scalability to LLMs and model interpretability. To address these issues, we conceptualize i.i.d. noise terms as token-level implicit causes—natural language expressions that reflect a speaker's underlying emotions, intentions, or situational context. Building on this insight, we introduce ICE (Implicit-Cause-Enhanced), an instruction-based framework that leverages implicit causes to enhance causal reasoning in LLMs. First, we design prompts that heuristically guide LLMs to generate implicit causes, which are then iteratively refined via an external evaluation mechanism. Second, by incorporating these implicit causes as intermediate reasoning steps, ICE improves the accuracy of emotion-cause pair prediction. Moreover, we distill the rationales produced by ICE into lightweight generative models, demonstrating that even small models can benefit from implicit-cause-driven reasoning. Extensive experiments in both instruction-based and distillation-based settings confirm the effectiveness, robustness, and interpretability of our approach.

Evaluating gpt-4 for zero-shot classification of bleeding and clotting events: Can large language models serve as second reviewers?