Large language models powered system safety assessment: applying STPA and FRAM

Uncertainty treatment in risk analysis of complex systems: The cases of STAMP and FRAM

The functional resonance analysis method (FRAM) is a system-based method to understand highly complex sociotechnical systems. Besides learning from safety occurrences or undesirable states, FRAM can be used to understand how things go well in a system, by identifying gaps between “work as imagined” (WAI) and “work as done” (WAD). FRAM is increasingly used in many domains and can enhance our understanding of a complex system and proposes strategies to refine the work design. This systematic review identified 108 FRAM research papers from 2006–2019. Most of these papers were conducted by European researchers and employed qualitative methods such as document analysis, interviews, and focus groups with subject matter experts (SMEs) and observations to develop WAI and WAD. Despite being used in healthcare, construction, and maritime sectors among others, aviation was the most commonly explored domain in FRAM studies. The 26 FRAM studies in aviation explored many aspects of the aviation industry, including Air Traffic Control (ATC) systems, cockpit operation, ground handling, maintenance, and a range of past safety incidents, like runway incursions. This paper also characterises the FRAM studies focused on aviation in terms of the common methods and steps used to build FRAM and the available software tools to build FRAM nets. Current FRAM illustrates its advantages in capturing the dynamic and nonlinear nature of complex systems and facilitates our understanding and continual improvement of complex systems. However, there are some critical issues in FRAM use and interpretation, such as the consistency of methods and complexity and reliability of data collection methods, which should be considered by researchers and FRAM users in industry.

System safety assessment using safety entropy

Quality improvements (QIs) in dynamic and complex health care contexts require resilience and take variability into account in quality improvement. The Functional Resonance Analysis Method (FRAM) helps us understand resilience and gain insight into (un)desirable variability in the complex system of daily practice. We explored how using FRAM in the Deming cycle of a QI project can help professionals and researchers learn from, reflect upon, and improve complex processes. We used FRAM in a Dutch hospital to study a QI: Critical Care Outreach Service (CCOS). The aim was to use FRAM before and after implementation to create a FRAM model and reflect to health care professionals the mismatch between Work As Imagined (WAI) and Work As Done (WAD). The WAI FRAM model was co-created with professionals before the implementation of CCOS. We used descriptions of tasks and processes for ICU nurses and verified them in 30-minute semistructured interviews (N = 2). WAD was created by input of semistructured interviews with key professionals in CCOS (N = 21) and 3 nonparticipant observations of trained CCOS nurses. We validated WAD in 2 dialogue sessions with key professionals (N = 11). Data collection continued until saturation. Juxtaposing the WAI and WAD models showed that WAD contained additional functions and highlighted unexpectedly complex functions. Reflecting on the application of FRAM with health care professionals revealed opportunities and challenges, especially time investment. FRAM helps professionals outline processes and tasks (WAI), learn from, and reflect upon their daily practice (WAD). FRAM models help professionals identify variability proactively to improve practices that enhance resilient performance.

Aim/Purpose: The study investigates the factors influencing the acceptance and utilisation of large language models (LLMs) (predictor variables of LLM usage), such as ChatGPT, in Learning design by instructional designers and university-teaching academics from various countries. Background: Large language models (LLMs) have exploded onto the scene, transforming the landscape of learning design. Instructional designers and university teaching academics have been overburdened with content creation for their teaching programmes, and the arrival of LLM models will help in this regard by developing more interactive content that drives student engagement and, in turn, contributes to student success. Since LLMs are a relatively new phenomenon, little is known about the factors influencing their acceptance in learning design; therefore, this research is needed, as learning design principles are the bedrock of student engagement and success. Methodology: A cross-sectional correlational quantitative study was employed. Data was collected using an online questionnaire posted on social media, including LinkedIn, from 203 instructional designers and university teaching academics. Purposive and snowball sampling methods were used to target instructional designers and university teaching academics at colleges and universities worldwide. Participants were asked to share the survey link with fellow instructional designers and university-teaching academics in their communities. The factor structure of the data was determined using exploratory factor analysis. Nonetheless, the factor structure derived from the LLMs did not entirely reflect the original configuration of the Unified Theory of Acceptance and Use of Technology (UTAUT3), as certain predictors appeared to coalesce, indicating LLMs’ unique nature in learning design. Confirmatory factor analysis was used to verify the fit of the data on the measurement model. First-order and second-order structural modelling were used to identify the structural relationships among the variables. Contribution: The study determines significant factors for the acceptance of LLMs by instructional designers and academic teaching staff in learning design, enabling possible opportunities for best practices in the field through interventions to optimize LLM usage. The study applies the technology acceptance model to the emerging LLM technology and extends the technology acceptance model by adding the trust construct as a predictor variable. Findings: The structural analysis results indicated that the ingrained LLM practices, LLM peer-driven expectations, innovative propensity towards LLM adoption, reliability and provider trust in LLMs, and ease of use and support influenced perceived LLM benefits and usage, but community standards and infrastructure had no influence. The second-order structural equation modelling indicated that perceived LLM benefits and usage and ingrained LLM habits contributed most to the learning design. Recommendations for Practitioners: Teaching academics and instructional designers must use LLMs in designing content, assessments, and interactive learning activities, and attend LLM training workshops on prompting and best practices in integrating LLMs into learning and teaching to see their benefits; hence, regular use of LLMs will then lead to trust and innovation in LLMs usage, enhancing learning design and improving student learning outcomes. Recommendation for Researchers: Researchers must use mixed methods approaches to have a deeper understanding of the factors influencing LLMs. Since habit and perceived LLM benefits and usage contributed the most variance to learning design, researchers must investigate strategies that optimise these factors in learning design, such as effective intervention strategies that can help form positive LLM habits. In addition, the findings provide researchers with a starting point for future research. Further researchers must investigate interventions that optimise the influence of personal innovativeness and trust that contributed the least variance to learning design, hence unlocking the potential of LLMs in learning design through innovation, responsible, and ethical use. Impact on Society: The use of LLMs in learning design has a high possibility of transforming education, specifically the learning design landscape. Using LLMs will free up more time for teaching academics and instructional designers so that they spend more time on higher-order thinking skill demands. Consequently, the students will be exposed to more engaging and interactive content, resulting in improved learning outcomes. Future Research: Future research must include context-derived external variables in technology acceptance models, such as levels of prompting competencies, to provide a deeper understanding of LLMs. In addition, future research must be based on the application and impact of LLMs on student engagement and success, and their attainment of 21st-century skills.

The introduction of the maritime autonomous surface ship (MASS) to the maritime industry will open up a new era and bring about a new paradigm shift in terms of cost efficiency, maritime accidents, and human resources. Various studies are currently being conducted to realize the MASS. Understanding the scope and direction of these studies will be of great help for future MASS research. In this study, the current development status of technologies for autonomous ships is identified, and considerations and directions of improvement are suggested for six major research fields that cover all technological issues of MASS. Firstly, the results of the regulatory scoping exercise (RSE) on the International Maritime Organization (IMO) conventions to accept MASSs are identified; in particular, human elements are identified as vital issues to be considered for the design and operation of MASSs. Secondly, various studies on the decision-making system are identified, and the future direction is suggested. Thirdly, in terms of ship design and propulsion system, design changes for autonomous cargo ships are investigated, with their potential impacts to be considered. Fourthly, the communication system will need to be robust and supported by multiple systems to minimize potential risk with third-party infrastructures, and suitable protection of systems, networks, and data will be required as an integral part of the safety system for cybersecurity. Fifthly, issues of maintenance and repair are identified, with a maintenance strategy to be considered. Lastly, hazard analysis of the autonomous ship is explored, and system-theoretic process analysis (STPA) and the functional resonance analysis method (FRAM) are identified as the most representative new methods that can be used for hazard analysis of autonomous ships.

Evaluating gpt-4 for zero-shot classification of bleeding and clotting events: Can large language models serve as second reviewers?

Purpose This study aims to explore the integration of large language models (LLMs) and vision-language models (VLMs) in robotics, highlighting their potential benefits and the safety challenges they introduce, including robustness issues, adversarial vulnerabilities, privacy concerns and ethical implications. Design/methodology/approach This survey conducts a comprehensive analysis of the safety risks associated with LLM- and VLM-powered robotic systems. The authors review existing literature, analyze key challenges, evaluate current mitigation strategies and propose future research directions. Findings The study identifies that ensuring the safety of LLM-/VLM-driven robots requires a multi-faceted approach. While current mitigation strategies address certain risks, gaps remain in real-time monitoring, adversarial robustness and ethical safeguards. Originality/value This study offers a structured and comprehensive overview of the safety challenges in LLM-/VLM-driven robotics. It contributes to ongoing discussions by integrating technical, ethical and regulatory perspectives to guide future advancements in safe and responsible artificial intelligence-driven robotics.

Readability & quality of large language model responses in CAR-T patient education

The Functional Resonance Analysis Method (FRAM) is extensively used to qualitatively analyze the risk of socio-technical systems. To quantitatively assess system risk, previous studies have explored various approaches to integrate FRAM with Bayesian Networks (BN). However, the process of mapping FRAM to BN relies heavily on subjective judgments, often lacking traceability and repeatability. In this paper, a standardized method for mapping FRAM to BN is proposed through introducing Accimap. Firstly, each FRAM’s functional hexagon is transformed into an Accimap, with the six aspects of functions corresponding to Accimap factors. In addition, their internal coupling relationships are obtained by cause and effect investigation in Accimap. Secondly, multiple Accimaps are connected based on the upstream-downstream couplings between functions in FRAM. Through this method, (i) the aspects of FRAM are transformed into Accimap factors and then BN nodes, and (ii) the couplings among the six aspects of functions and across multiple hexagons are converted into BN directed edges.

Over the past two decades, systemic-based risk assessment methods have garnered more attention, and their use and popularity are growing. In particular, the functional resonance analysis method (FRAM) is one of the most widely used systemic methods for risk assessment and accident analysis. FRAM has been progressively evolved since its starting point and is considered to be the most recent and promising step in understanding socio-technical systems. However, there is currently a lack of any formal testing of the reliability and validity of FRAM, something which applies to Human Factors and Ergonomics research as a whole, where validation is both a particularly challenging issue and an ongoing concern. Therefore, this paper aims to define a more formal approach to achieving and demonstrating the reliability and validity of an FRAM model, as well as to apply this formal approach partly to an existing FRAM model so as to prove its validity. At the same time, it hopes to evaluate the general applicability of this approach to potentially improve the performance and value of the FRAM method. Thus, a formal approach was derived by transferring both the general understanding and definitions of reliability and validity as well as concrete methods and techniques to the concept of FRAM. Consequently, predictive validity, which is the highest maxim of validation, was assessed for a specific FRAM model in a driving simulator study using the signal detection theory. The results showed that the predictive validity of the FRAM model is limited and a generalisation with changing system conditions is impossible without some adaptations of the model. The applicability of the approach is diminished because of several methodological limitations. Therefore, the reliability and validity framework can be utilised to calibrate rather than validate an FRAM model.

The paper studies the problem of software code migration during the transition from the old version of the SAP ERP 6.0 system to modern ERP platforms, in particular SAP S/4HANA. The necessity of automation of the migration process using artificial intelligence methods and tools is substantiated. A comprehensive analysis of existing academic and industrial case studies confirmed the feasibility of using large language models for code migration. A number of advantages of the approach based on large language models are revealed, in particular, the ability to handle various migration scenarios. The key risks and limitations common in the enterprise environment are highlighted, including data security, privacy, high cost of API calls and limited knowledge of models regarding the subject area. A method is developed for selecting a large open source language model that can be safely used in the enterprise landscape, thereby mitigating the risks associated with cloud solutions. The method is based on the developed system of criteria, including: model size, context window size, license type, support for fine-tuning training, focus on code quality (measured using a reference dataset), support for model quantization and community support. The hybrid AHP-TOPSIS method is proposed for systematic ranking of candidate models. The AHP method was used to check the weights of the criteria, while TOPSIS was used to rank the models based on their proximity to the ideal solution. Based on the developed method, three large language models were selected: Qwen 2.5 Coder 14B, DeepSeek-Coder-V2 16B and Llama 3.1 8B. A comprehensive testing plan for the selected models was developed for further experimental studies. The scientific novelty of the work lies in the development of a method for selecting large language models for custom code migration tasks, which differs from the existing system of model selection criteria and the use of a hybrid AHP-TOPSIS approach for ranking candidate models. The practical value of the work is that the developed method allows you to select a model taking into account the constraints of the corporate environment and information security requirements, which will increase the level of automation in the migration of AVAP when switching to new versions of ERP systems from SAP SE. The developed method can be applied to the selection of large language models when upgrading any large computer systems developed in both open source and proprietary programming languages.

LLMs application to clinical practice is growing fast. However, LLMs are less studied in geriatrics practice but are urgently needed. This symposium will address whether LLMs allocation to geriatric practice can be trusted via five approaches. 1) LLMs generated gender and race-biased outputs. We will demonstrate whether LLMs generated age-biased output by assessing their geriatric attitude evaluated by social workers. 2). LLMs passed USMLE and other examinations. We will demonstrate whether LLMs can pass geriatrics knowledge competence tests evaluated by geriatricians 3). LLMs performed well on clinical vignettes from different clinical disciplines. We will demonstrate whether LLMs can perform well on geriatrics 5M-based vignettes of older adults evaluated by clinical providers and trainees 4) LLMs reviewed and summarized clinical charts. We will demonstrate whether LLMs can review geriatrics and general medicine notes to extract Mobility (one of Geriatrics 5Ms) documentation evaluated by geriatricians 5). LLMs can generate deprescribing recommendations, tapering schedules, and patient education materials. We will demonstrate their accuracy, safety, and appropriateness compared to recommendations from a multidisciplinary team of pharmacists, geriatricians, and nurses. Specifically, this symposium will address the following topics: 1) Geriatric Attitude of ChatGPT4.o and Its Evaluation by Social Workers. 2) ChatGPT4.o Geriatrics Knowledge Competency and Its Evaluation by Geriatricians. 3) LLMs application to geriatrics 5Ms evaluated by clinical providers and trainees. 4) Using LLMs to Extract and Assess Mobility Documentation for Age-Friendly Health System evaluated by geriatricians. 5) Using LLMs to generate medication deprescribing recommendations compared to clinician-led deprescribing recommendations.

Introduction Advances in large language models (LLMs) have opened new avenues for healthcare applications. Recently, ChatGPT-4 successfully achieved the pass mark >80% in 5 of 10 sleep medicine examination domains, indicating a strong foundational knowledge of sleep medicine. However, the ability to answer USMLE-type multiple-choice questions may not equate to the capacity to offer accurate and comprehensive answers to clinical queries or real-world case scenarios. Current literature exhibits a significant gap in validation research examining the clinical utility of LLMs in evidence-based medicine practice. This investigation aims to evaluate the potential usefulness and reliability of LLMs as adjunctive clinical decision-support tools (namely, “curbside consultants”) in sleep medicine. Methods Six clinical sleep queries and six case scenarios were presented to 6 LLMs including 3 general-purpose LLMs (ChatGPT-4o, Gemini-1.5-Pro, and Llama-3.1-405B) and 3 medical-specialized LLMs (OpenEvidence, Clara AI, and MediGPT). Performance assessment was conducted independently utilizing two independently developed 5-point Likert scales evaluating two primary domains: answer content (accuracy, relevance, comprehensiveness/depth, clarity/coherence, and unique insightfulness) and reference quality (accuracy, relevance, currentness, comprehensiveness/depth, and searchability). Benchmark answers were established through consensus among four sleep medicine specialists. Analysis of variance was used to compare the performance of the LLMs. Results The medical LLMs demonstrated superior overall performance compared to the general LLMs (P< 0.001). The primary distinction was observed in reference quality metrics, where medical LLMs significantly outperformed general LLMs across all parameters: accuracy, relevance, and searchability (p< 0.001), currentness (p=0.001), and comprehensiveness/depth (p=0.010). Notably, Open Evidence achieved the highest reference quality (p< 0.001). In contrast, the answer content analysis revealed no significant overall differences between medical and general LLMs (p=0.659). Most answer content-related metrics, including accuracy, relevance, or unique insightfulness, did not differ significantly (p>0.050). An exception was clarify/coherence, where medical LLMs were superior to general LLMs (p=0.030). Furthermore, MediGPT and ChatGPT-4o displayed better content comprehensiveness/depth relative to other LLMs (p< 0.001). These two LLMs exhibited comparable performance in overall metrics (p=0.615), answer contents (p=0.922), and reference quality (p=0.621). Conclusion Both medical-specialized and general-purpose LLMs show promise as adjunctive decision-support tools in clinical practice. However, substantial improvements in reference quality are critically needed across most LLM platforms. Support (if any)

A Monte Carlo evolution of the Functional Resonance Analysis Method (FRAM) to assess performance variability in complex systems