Prompt Driven Test Generation: Leveraging Large Language Models and Knowledge Graphs for Quality Assurance in Data Intensive Software System

In recent years, the landscape of digital biomedicine and healthcare has been reshaped due to the disruptive breakthroughs in AI-facilitated by tremendous data and high-performance computers, large language models (LLMs) have transformed information technology from accessing data to performing analytical tasks. While demonstrating unprecedented capabilities, LLMs have been found unreliable in tasks requiring factual knowledge and rigorous reasoning. Biomedicine and healthcare, as an important vertical domain rapidly benefitting from progress in AI, necessitates strict requirements on the accuracy, controllability, and interpretability of analytical models, posing critical challenges for LLMs. Despite recent studies addressing the hallucination problem of LLMs, research on empowering LLMs with the ability to plan, reason, and ground with explicit knowledge has also started to prosper, especially in the biomedicine and healthcare domain. On the other hand, biomedical data are enormous and notoriously complex, coming from various sources (e.g., biomedical knowledge bases, online literature, and hospitals) and bearing various modalities (e.g., tables, texts, images and time-series). Healthcare professionals have spent decades collecting, cleaning, and curating various types of data. The processes are extremely costly, producing various datasets with different data schemas, coding systems, and quality standards, many privately owned by the creators, making their integrative analysis and utilization through unified AI techniques still rather challenging. The generalizability of LLMs across different types of data endow them strong promises in automating the processing of large-scale complex healthcare data such as into unified knowledge graphs (KGs). Our goal in this survey is to systematically investigate and summarize recent studies on the unification of LLMs and KGs, towards fully utilizing the value of complex data, unleashing the power of generative AI, and expediting next-generation AI for biomedicine and healthcare applications.

In recent years, Large Language Models (LLMs) have rapidly advanced in language understanding, reasoning, and generation, and are increasingly adopted as the “brain” of industrial intelligent systems. Nevertheless, in high‑risk and strongly regulated domains they still exhibit hallucination, weak domain grounding, limited interpretability, and privacy as well as security constraints. Knowledge graphs (KGs) encode domain entities, relations, rules, and events explicitly, providing controllable semantics and an explainable reasoning substrate. Retrieval‑augmented generation (RAG) injects external evidence into LLM prompting, while GraphRAG further introduces graph indexing and community‑level retrieval to preserve global structure and support multi‑hop reasoning. This review summarizes the evolution of LLMs, KG modeling and extraction, GraphRAG mechanisms, and a general fusion framework. Typical industrial applications are surveyed, and a coal mine flood emergency plan generation and evaluation approach is discussed to illustrate the practical value of graph‑grounded large models. KG‑enhanced retrieval also supports provenance tracking, allowing industrial users to audit the evidence behind model outputs.

The unification of large language models (LLMs) and knowledge graphs (KGs) has emerged as a hot topic. At the LLM+KG'24 workshop, co-located with VLDB 2024 in Guangzhou, China, the key theme explored was important data management challenges and opportunities due to the effective interaction between LLMs and KGs. The report outlines major directions and approaches presented by various speakers during the workshop.

Numerous large knowledge graphs, such as DBpedia, Wikidata, Yago and Freebase, have been developed in the last decade, which contain millions of facts about various entities in the world. These knowledge graphs have proven to be incredibly useful for intelligent Web search, question understanding, in-context advertising, social media mining, and biomedicine. As some researchers have pointed out, a knowledge graph is not just a graph database, but it should have a layer of conceptual knowledge, which is usually represented as a set of first-order rules. However, it is challenging to automatically extract first-order rules from large knowledge graphs. In particular, traditional models are usually unable to handle rule learning in large knowledge graphs. This chapter aims to present state-of-the-art techniques and models on learning first-order rules using representation learning. By first recalling some basics of rule learning in knowledge graphs, we will introduce useful techniques of embedding-based rule learning through major models such as RLvLR and TyRuLe, which embed paths in knowledge graphs into latent spaces for candidate rule search. Then we evaluate the rule learning efficiency and the quality of automatically learned rules by applying them in link prediction. Before the chapter is concluded, we will also discuss some future research problems in the area.

This paper focuses on building an innovative industrial financial risk rapid discovery model based on large language model and knowledge graph inference. Firstly, this work deeply analyzes the diversity, concealment and rapid propagation of industrial financial risks. Secondly, it points out the limitations of traditional risk assessment methods when dealing with massive and heterogeneous financial data. Then, it designs an integrated strategy to integrate the powerful natural language processing and understanding ability of large language models. Combined with the advantages of knowledge graph in structured information representation, this model analyzes financial text data through a large language model. Hence, key information can be automatically extracted and a preliminary risk knowledge base can be built. Finally, an industrial financial network can be established to realize correlation mining and dynamic evolution analysis of risk factors. The experimental results show that the model can effectively capture risk signals in the industrial chain, and detect potential financial risks in advance. This study not only enriches the theoretical framework of industrial financial risk management, but also provides strong technical support for risk prevention and control in practice.

Scheduling satellite tasks requires intricate coordination of numerous interconnected activities, posing challenges for ground station operators who struggle to decipher autonomous decision-making systems without explanatory responses. This paper advocates for explainable artificial intelligence to address this gap, leveraging large language models and knowledge graphs to enhance the transparency of scheduling logic. The study explores how operators can retrieve relevant information through natural language queries by integrating language processing with knowledge graphs as core data structures. This approach allows for manual access to knowledge graphs, supplemented with textual explanations, and enables operators to explore different scheduling scenarios, thus improving system robustness and adaptability. Additionally, this paper analyzes how current knowledge graphs and natural language processing techniques are utilized and how they can enhance explainability in satellite scheduling. It investigates the modeling of satellite schedules and environmental data within the knowledge graph. The study introduces a novel approach for generating categorized queries, executable code for knowledge graph data extraction via language modeling, and providing explanatory answers. These are evaluated for correctness, validity, and linguistic quality, demonstrating the effectiveness of this approach.

Machine learning algorithms for link prediction can be valuable tools for hypothesis generation. However, many current algorithms are black boxes or lack good user interfaces that could facilitate insight into why predictions are made. We present LinkExplorer, a software suite for predicting, explaining and exploring links in large biomedical knowledge graphs. LinkExplorer integrates our novel, rule-based link prediction engine SAFRAN, which was recently shown to outcompete other explainable algorithms and established black-box algorithms. Here, we demonstrate highly competitive evaluation results of our algorithm on multiple large biomedical knowledge graphs, and release a web interface that allows for interactive and intuitive exploration of predicted links and their explanations. A publicly hosted instance, source code and further documentation can be found at https://github.com/OpenBioLink/Explorer. Supplementary data are available at Bioinformatics online.

This study aims to construct a knowledge syst em for Traditional Chinese Patent Medicine (TCPM) based on Large Language Model (LLM) and Knowledge Graph (K G) to address the complexity of TCPM knowledge and its <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$c$</tex> linical application challenges. It seeks to solve issues inhere nt in traditional TCPM knowledge utilization, such as infor mation silos, retrieval difficulties, and the lack of effective decision support. By integrating multi-source heterogeneous data, including the Pharmacopoeia of the People's Republic of China and various medical information platforms, this re search utilizes an LLM for efficient knowledge extraction a nd fusion to build the Traditional Chinese Patent Medicine Knowledge Graph (TCPM-KG). Experimental results show <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$t$</tex> hat the model achieved an F1-score of 0.9553 in the Name d Entity Recognition(NER) task, indicating that LLM can ef fectively process complex texts in the TCPM domain and si gnificantly enhance the automation of knowledge extraction. To date, TCPM-KG contains 1,607 TCPM instances and 77, 100 knowledge triplets. The advantages of this study lie in <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$i$</tex> ts integration of multi-source heterogeneous data to create a KG with broad coverage and fine granularity, and its prop osal of a “KG+LLM” methodology that spans the entire co nstruction process. This work successfully demonstrates the feasibility and efficiency of using an LLM as a core techno logy for building a TCPM-KG, providing a new technical <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$p$</tex> ath and foundation for the modernization and intelligent de velopment of Traditional Chinese Medicine.

AIREG: Enhanced Educational Recommender System with Large Language Models and Knowledge Graphs

Conversational AI systems have emerged as key enablers of human-like interactions across diverse sectors. Nevertheless, the balance between linguistic nuance and factual accuracy has proven elusive. In this paper, we first introduce LLMXplorer, a comprehensive tool that provides an in-depth review of over 205 large language models (LLMs), elucidating their practical implications, ranging from social and ethical to regulatory, as well as their applicability across industries. Building on this foundation, we propose a novel functional architecture that seamlessly integrates the structured dynamics of knowledge graphs with the linguistic capabilities of LLMs. Validated using real-world AI news data, our architecture adeptly blends linguistic sophistication with factual rigor and further strengthens data security through role-based access control. This research provides insights into the evolving landscape of conversational AI, emphasizing the imperative for systems that are efficient, transparent, and trustworthy.

Complexity or Patterns? Designing Neuro-Symbolic Reasoning with Large Language Models and Knowledge Graphs

Knowledge Graphs (KGs) represent heterogeneous domain knowledge on the Web and within organizations. There exist shapes constraint languages to define validating shapes to ensure the quality of the data in KGs. Existing techniques to extract validating shapes often fail to extract complete shapes, are not scalable, and are prone to produce spurious shapes. To address these shortcomings, we propose the Quality Shapes Extraction (QSE) approach to extract validating shapes in very large graphs, for which we devise both an exact and an approximate solution. QSE provides information about the reliability of shape constraints by computing their confidence and support within a KG and in doing so allows to identify shapes that are most informative and less likely to be affected by incomplete or incorrect data. To the best of our knowledge, QSE is the first approach to extract a complete set of validating shapes from WikiData. Moreover, QSE provides a 12x reduction in extraction time compared to existing approaches, while managing to filter out up to 93% of the invalid and spurious shapes, resulting in a reduction of up to 2 orders of magnitude in the number of constraints presented to the user, e.g., from 11,916 to 809 on DBpedia.

We describe the integration of a Large Language Model (LLM) and a knowledge graph (KG) in an application of medical informatics. The LLM is BioBERT, a pre-trained biomedical language representation model for biomedical text mining. The task, prediction of Type 2 Diabetes (T2D) in the UK Biobank cohort, a population-based prospective cohort study aimed at improving the prevention, diagnosis and treatment of a wide range of serious diseases, including diabetes. Patient history is represented as date-stamped ICD-10 codes, a hierarchical classification of diseases (the KG). The connection between the LLM and the KG is precisely the shared medical vocabulary used in the ICD-10 descriptions and the resources used in fine tuning the LLMs. We explore the impact of the various levels of the KG hierarchy (subcategories, categories, blocks, and chapters) on the performance of classifiers developed for T2D.

Smart city systems are increasingly built with data analytics and machine learning techniques, basing on massive data sets. They have a heavy impact on human behavior and quality of life, and thus need to deliver a controllable and sufficiently transparent experience for the users.The aim of my work is to make smart city systems more interoperable and explainable, involving data visualization and communication techniques, sensor data processing, as well as the associated intelligent data value chain production and consumption. Here, the data and the information are shared employing Knowledge Graphs, that are becoming a key enabler for large-scale processing of massive collections of interrelated facts. Examples include the Google Knowledge Graph with dozens of billion facts, dataCommons, DBPedia, YAGO, and Knowledge Vault, a very large scale probabilistic knowledge graph created with information extraction methods for unstructured or semi-structured information. Specifically, Knowledge Graphs provide the means of development of the newest methods for data management, data fusion, data merging, and graph and network optimization and modeling, serving as a source of high quality data and a base for information integration.In particular, Knowledge Graphs help to infer new relationships out of existing facts, giving context and meaning to the content, and can be used in applications. For example, the data generated by a computer vision system could be semantically represented and shared across numerous systems, taking into account the needs and requirements of these systems, as well as the context, provenance, licensing and consent aspects of the generated data. I demonstrate Knowledge Graphs-based methods in advanced smart city applications from the domains such as automation and construction of buildings, energy efficiency, tourism, transport.

Currently knowledge graph has been widely applied to various fields. Although the data scale is large, there is still a lot of useful but implicit information in it. Thus, a powerful reasoning system is required to derive these data. However, current reasoning systems cannot accomplish this task very well. On the one hand, stand-alone reasoning systems cannot meet the demand of large data. On the other hand, the reasoning ability of existing distributed reasoning systems is limited because of the lack of expressive inference rules. In this paper, we propose and implement a distributed reasoning system KGRL for knowledge graph based on OWL2 RL. It has a more powerful reasoning ability due to more expressive rules. It also supports optimization for redundant data. Besides, a rule-based algorithm is designed to find the inconsistent data. Experimental results show that KGRL can derive more implicit information efficiently compared to other reasoning systems. Moreover, KGRL is capable of eliminating redundant data, which can reduce the storage of knowledge graph by an average of 42%. Finally, KGRL also performs well for the detection of inconsistencies in knowledge graph.