ADKGD: Anomaly Detection in Knowledge Graphs with Dual-Channel Training

Extracting information from unstructured and semi-structured geoscience literature is a crucial step in conducting geological research. The traditional machine learning extraction paradigm requires a substantial amount of high-quality manually annotated data for model training, which is time-consuming, labor-intensive, and not easily transferable to new fields. Recently, large language models (LLMs) (e.g., ChatGPT, GPT-4, and LLaMA), have shown great performance in various natural language processing (NLP) tasks, such as question answering, machine translation, and text generation. A substantial body of work has demonstrated that LLMs possess strong in-context learning (ICL) and even zero-shot learning capabilities to solve downstream tasks without specifically designed supervised fine-tuning. In this paper, we propose utilizing LLMs for geoscience literature information extraction. Specifically, we design a hierarchical PDF parsing pipeline and an automated knowledge extraction process, which can significantly reduce the need for manual data annotation, assisting geoscientists in literature data mining. For the hierarchical PDF parsing pipeline, firstly, a document layout detection model fine-tuned on geoscience literature is employed for layout detection, obtaining layout detection information for the document. Secondly, based on the document layout information, an optical character content parsing model is used for content parsing, obtaining the text structure and plain text corresponding to the content. Finally, the text structure and plain text are combined and reconstructed to ultimately obtain the parsed structured data. For the automated knowledge extraction process, firstly, the parsed long text is segmented into paragraphs to adapt to the input length limit of LLMs. Subsequently, a few-shot prompting method is employed for structured knowledge extraction, encompassing two tasks: attribute value extraction and triplet extraction. In attribute value extraction, prompts are generated automatically by the LLMs based on the subdomain and attribute names, facilitating the location and extraction of values related to subdomain attribute names in the text. For triplet extraction, the LLMs employ a procedural approach to entity extraction, entity type extraction, and relation extraction, following the knowledge graph structure pattern. Finally, the extracted structured knowledge is stored in the form of knowledge graphs, facilitating further analysis and integration of various types of knowledge from the literature. Our proposed approach turns out to be simple, flexible, and highly effective in geoscience literature information extraction. Demonstrations of information extraction in subdomains such as radiolarian fossils and fluvial facies have yielded satisfactory results. The extraction efficiency has significantly improved, and feedback from domain experts indicates a relatively high level of accuracy in the extraction process. The extracted results can be used to construct a foundational knowledge graph for geoscience literature, supporting the comprehensive construction and efficient application of a geoscience knowledge graph.

Knowledge graphs (KGs) have transformed data management within the manufacturing industry, offering effective means for integrating disparate data sources through shared and structured conceptual schemas. However, harnessing the power of KGs can be daunting for non-experts, as it often requires formulating complex SPARQL queries to retrieve specific information. With the advent of Large Language Models (LLMs), there is a growing potential to automatically translate natural language queries into the SPARQL format, thus bridging the gap between user-friendly interfaces and the sophisticated architecture of KGs. The challenge remains in adequately informing LLMs about the relevant context and structure of domain-specific KGs, e.g., in manufacturing, to improve the accuracy of generated queries. In this paper, we evaluate multiple strategies that use LLMs as mediators to facilitate information retrieval from KGs. We focus on the manufacturing domain, particularly on the Bosch Line Information System KG and the I40 Core Information Model. In our evaluation, we compare various approaches for feeding relevant context from the KG to the LLM and analyze their proficiency in transforming real-world questions into SPARQL queries. Our findings show that LLMs can significantly improve their performance on generating correct and complete queries when provided only the adequate context of the KG schema. Such context-aware prompting techniques help LLMs to focus on the relevant parts of the ontology and reduce the risk of hallucination. We anticipate that the proposed techniques help LLMs to democratize access to complex data repositories and empower informed decision-making in manufacturing settings.

Large Language Models (LLMs) face significant challenges when attempting to utilize knowledge not encoded during pre-training. In-context learning (ICL) addresses this limitation by embedding relevant information directly into prompts, enabling LLMs to leverage external knowledge without updating their parameters. Recent advances have explored integrating knowledge graph (KG) information into prompts, taking advantage of the structured and factual representations of entities and relationships that KGs provide. A common approach involves identifying key concepts in a task, grounding them to KG nodes, extracting ego-subgraphs centered on these concepts, and incorporating them into prompts. However, not all subgraphs are equally relevant to the task, and selecting appropriate knowledge remains a critical challenge, as irrelevant or noisy subgraphs can reduce model accuracy. This study investigates how KG-based information related to task-specific concepts influences LLM performance on multiple-choice question answering (MCQA) tasks. We implement a four-stage pipeline that (1) identifies key concepts from questions and grounds them in the KG, (2) extracts individual ego-subgraphs for each concept, (3) integrates these subgraphs into prompts, and (4) evaluates their impact on LLM reasoning via probability scoring. Our analysis highlights which concept-specific subgraphs enhance performance, which introduce misleading information, and which have neutral effects. We show that carefully selected KG subgraphs can substantially outperform others in semantic relevance. Furthermore, we examine how the size and connectivity of selected KG subgraphs influence model performance. Overall, this work deepens understanding of KG-based knowledge selection in ICL and informs the design of more effective, targeted prompting strategies. This project's source code is publicly available at https://github.com/maryam-ghanbari/InformativenessOfKGs.

In today’s rapidly evolving business landscape, Artificial Intelligence (AI), and specifically Large Language Models (LLMs), are redefining how organizations operate, make decisions, and engage with customers. AI-driven technologies have become indispensable, providing businesses with powerful tools to streamline operations, derive actionable insights from vast data, and foster more meaningful customer interactions. For business leaders, scholars, and practitioners alike, understanding the transformative potential of AI isn’t just advantageous—it’s essential to staying competitive in an increasingly data-driven world.This editorial delves into recent scholarly advancements in LLM applications within business contexts, analyzing studies that explore AI’s potential across various domains, from decision support to creative industries. By introducing a structured framework, this editorial highlights key insights and contributions from recent studies, assessing their value to academia and industry. The following comparative analysis sheds light on how these innovations shape our understanding of AI’s role in business while pointing to future research directions.Puyt and Madsen's (2024) study stands out as a foundational exploration of LLM accuracy, assessing ChatGPT-4's ability to recount the history of the SWOT analysis-a vital business strategy tool. Their findings reveal that, while ChatGPT-4 effectively conveys general concepts, it struggles with detailed historical information, often producing inaccuracies or "hallucinations." This gap underscores the need for LLMs to be trained with verified academic data, particularly for strategic business applications that demand precision. This study not only contributes to the literature by proposing methods to evaluate AI accuracy in historical contexts but also highlights the importance of rigorous information vetting in industry settings where reliability is crucial.In contrast, Raikov et al. (2024) explore a hybrid intelligence model that combines LLM capabilities with explainable AI (XAI) principles to enhance human-machine collaboration. Their approach emphasizes cognitive semantics, improving transparency and decision-making efficiency. The hybrid model's real-time adaptability addresses the needs of complex, regulated industries such as finance and healthcare, where trust in AI decisions is paramount. Academically, this study provides a valuable addition to XAI literature by demonstrating how LLMs can bridge the gap between AI autonomy and human oversight, making it a model for future human-AI interactions in complex business environments.Another significant study by Mariotti and colleagues (2024) examines the integration of LLMs with enterprise knowledge graphs to enhance data-driven decision-making. By enabling organizations to leverage knowledge graphs for more accurate and scalable data retrieval, this research provides a robust framework for businesses seeking efficient knowledge management systems. The academic contribution here lies in advancing the dialogue between LLMs and knowledge graphs, emphasizing ethical data handling and quality standards essential for industry applications. For enterprises, the study offers practical solutions to achieve streamlined data management, balancing automation with privacy and security. 2024) take a different approach, investigating LLMs' role in creative industries, specifically within fashion design. They introduce a hybrid intelligence model that supports creative processes, allowing AI to complement rather than replace human ingenuity. While LLMs in this field demonstrate potential in automating repetitive design tasks and enhancing customer personalization, the study reveals limitations in AI's ability to handle spatial and stylistic nuances. This study's academic contribution lies in promoting human-AI co-creation, inspiring further research into AI applications across diverse creative sectors, including media and marketing.Collectively, these studies not only illuminate LLMs' transformative potential in business but also highlight critical ethical and operational considerations. Ensuring accuracy, transparency, and data privacy are vital to responsibly integrating AI into business workflows. Future research should focus on enhancing LLM accuracy, refining hybrid intelligence models, and exploring creative AI applications, all while maintaining ethical standards. As LLMs evolve, interdisciplinary collaborations will be essential to harness their full potential, making AI an ethical, effective, and innovative force in the business world.

This paper investigates the impact of knowledge extraction methods on the effectiveness of RAG (Retrieval-Augmented Generation) systems that utilize knowledge graphs. It highlights that the quality of the knowledge graph, which is formed using various knowledge extraction methods, is crucial for overcoming limitations of large language models (LLMs), such as “hallucinations”. The paper analyzes the architectures of LightRAG and GraphRAG, emphasizing that the selection of an optimal knowledge extraction strategy depends on specific tasks and the subject area. LLMs have advanced significantly, but they have limitations, including generating factually incorrect information (“hallucinations”) and possessing “outdated knowledge”. RAG systems were proposed to address these issues by combining LLMs with external knowledge bases. This approach reduces hallucinations, ensures factual accuracy, solves the problem of outdated knowledge, and increases transparency. Knowledge graphs are powerful tools for structuring information, consisting of entities (nodes) and relations (edges). They enhance RAG systems by enabling more precise and contextually grounded retrieval compared to keyword-based searches. The quality of a knowledge graph depends on the knowledge extraction methods used, which include named entity recognition (NER), relation extraction (RE), entity linking (EL), and event extraction. Different methods, such as rule-based, classical machine learning, and deep learning approaches, have varying trade-offs in terms of accuracy, completeness, and scalability. Entity linking and knowledge graph completion are also crucial for accuracy and richness. LightRAG and GraphRAG are two main graph-based RAG systems. LightRAG uses the knowledge graph as a quick reference, requiring high precision in knowledge extraction to avoid noise degradation. GraphRAG uses the knowledge graph as a domain model, where completeness of extracted knowledge is more critical, though systematic errors are still harmful. Both systems rely on LLMs for knowledge extraction, which makes them dependent on the LLM’s quality and the size of document fragments processed. The theoretical analysis confirms that the effectiveness of RAG systems is critically dependent on knowledge extraction methods. The quality, completeness, and accuracy of the knowledge graph directly influence the RAG system’s ability to provide relevant, accurate, and truthful answers. Different RAG system architectures like LightRAG and GraphRAG have distinct requirements for knowledge graph characteristics, prioritizing either accuracy or completeness in knowledge extraction.

Recommender systems are increasingly used in many fields to support personalised and adaptive decision-making. Large Language Models (LLMs) like GPT and Knowledge Graphs (KGs) have sparked interest in combining their strengths to improve recommendation quality, context, and explainability. Although recent reviews discuss the benefits of KGs for cloud-based LLMs, they rarely focus on their use in recommender systems. This literature review fills that gap by examining how KGs and LLMs work together to enhance recommendations. This study were guided by three research questions on application areas, integration methods, and evaluation method. Additionally, the study uses the PRISMA approach to select and analyse 20 relevant papers. The results compare different methods and performance metrics, including precision, recall, NDCG, and F1-score. This review offers a focused summary of current progress and points out important directions for future research on KG-LLM recommender systems.

Knowledge graphs are a powerful way to represent and organize complex knowledge. They are used in many fields, like healthcare and finance. They allow for more insightful decision-making and discoveries. However, the quality of knowledge graphs depends heavily on their sources. Current methods for evaluating these sources are often slow and not scalable. They struggle to keep up with the large amount of online information. We created a new tool to address this problem. Our tool uses Large Language Models (LLMs) to assess online sources quickly. It evaluates websites based on credibility, relevance, content quality, coverage, comprehensiveness, and accessibility. We tested our tool on Halal tourism websites in Japan. We compared LLM evaluations with human expert judgments. Our comprehensive analysis revealed that certain LLM models, particularly GPT-3.5-turbo, GPT-4, and Mixtral-8x7B-Instruct-v0.1, showed strong correlation with human evaluations. Using a temperature setting of 0.4, these models demonstrated consistent and reliable performance across multiple evaluation runs. Our structured evaluation framework, incorporating weighted criteria validated through both expert input and statistical analysis, provides a robust foundation for automated source assessment. While some models showed varying performance across different criteria, our findings suggest that careful model selection and potential ensemble approaches could optimize evaluation accuracy. Our work contributes significantly to improving knowledge graph construction by demonstrating the viability of LLM-based source evaluation, while also identifying key areas for future research in scalability, cross-domain validation, and automated optimization.

In recent years, entertainment content, such as movies, music, and anime, has been gaining attention due tothe stay-at-home demand caused by the expansion of COVID-19. In the content domain, research in the field ofknowledge representation is primarily concerned with accurately describing metadata. Therefore, different knowledgerepresentations are required for applications in downstream tasks. In this study, we aim to clarify effectiveknowledge representation for predicting users’ latent preferences through a case study of an anime recommendationtask. We developed hypotheses from both quantitative and qualitative aspects on how to represent work knowledgeto improve recommendation performance, and verified them by changing the structure of knowledge representationaccording to the hypothesis. Initially, we constructed a Knowledge Graph (KG) by integrating domain-specific andgeneral-purpose data sources through the process of entity matching and imposing constraints on the properties.Subsequently, we constructed multiple KGs by varying the knowledge configuration. Specifically, we changed thecomposition of the data sources considered in the KG construction or excluded a triplet associated with an arbitraryproperty. After that, we fed the constructed KGs into the graph neural network recommender model and comparedthe recommendation performance. As a result, it was shown that the recommendation performance based on the KGcomposed of multiple data sources was the best, thus supporting the hypothesis from a quantitative aspect. Next,an ablation study on the properties revealed that knowledge characterizing the work itself contributed to the recommendationperformance, thus supporting the hypothesis from a qualitative aspect. Furthermore, we constructed atext-based KG by generating a new vocabulary from the “synopsis” text. It can describe the work’s storyline andworldview in more detail. We take it as an input to a Large Language Model (LLM) and extend the existing metadatabasedKG. The results showed that the KG considering both metadata and text had the best overall recommendationperformance, again confirming the hypothesis.

Knowledge graphs exhibit a typical hierarchical structure and find extensive applications in various artificial intelligence domains. However, large-scale knowledge graphs need to be completed, which limits the performance of knowledge graphs in downstream tasks. Knowledge graph embedding methods have emerged as a primary solution to enhance knowledge graph completeness. These methods aim to represent entities and relations as low-dimensional vectors, focusing on handling relation patterns and multi-relation types. Researchers need to pay more attention to the crucial feature of hierarchical relationships in real-world knowledge graphs. We propose a novel knowledge graph embedding model called Hierarchy-Aware Paired Relation Vectors Knowledge Graph Embedding (HPRE) to bridge this gap. By leveraging the power of 2D coordinates, HPRE adeptly model relation patterns, multi-relation types, and hierarchical features in the knowledge graph. Specifically, HPRE employs paired relation vectors to capture the distinct characteristics of head and tail entities, facilitating a better fit for relational patterns and multi-relation scenarios. Additionally, HPRE employs angular coordinates to differentiate entities at various levels of the hierarchy, effectively representing the hierarchical nature of the knowledge graph. The experimental results show that the HPRE model can effectively learn the hierarchical features of the knowledge graph and achieve state-of-the-art experimental results on multiple real-world datasets for the link prediction task.

BackgroundIn recent years, large language models (LLMs) have shown promise in various domains, notably in biomedical sciences. However, their real-world application is often limited by issues like erroneous outputs and hallucinatory responses.ResultsWe developed the knowledge graph–based thought (KGT) framework, an innovative solution that integrates LLMs with knowledge graphs (KGs) to improve their initial responses by utilizing verifiable information from KGs, thus significantly reducing factual errors in reasoning. The KGT framework demonstrates strong adaptability and performs well across various open-source LLMs. Notably, KGT can facilitate the discovery of new uses for existing drugs through potential drug–cancer associations and can assist in predicting resistance by analyzing relevant biomarkers and genetic mechanisms. To evaluate the knowledge graph question answering task within biomedicine, we utilize a pan-cancer knowledge graph to develop a pan-cancer question answering benchmark, named pan-cancer question answering.ConclusionsThe KGT framework substantially improves the accuracy and utility of LLMs in the biomedical field. This study serves as a proof of concept, demonstrating its exceptional performance in biomedical question answering.

The increasing reliance on Large Language Models (LLMs) for health information seeking can pose severe risks due to the potential for misinformation and the complexity of these topics. This paper introduces KnowNet a visualization system that integrates LLMs with Knowledge Graphs (KG) to provide enhanced accuracy and structured exploration. Specifically, for enhanced accuracy, KnowNet extracts triples (e.g., entities and their relations) from LLM outputs and maps them into the validated information and supported evidence in external KGs. For structured exploration, KnowNet provides next-step recommendations based on the neighborhood of the currently explored entities in KGs, aiming to guide a comprehensive understanding without overlooking critical aspects. To enable reasoning with both the structured data in KGs and the unstructured outputs from LLMs, KnowNet conceptualizes the understanding of a subject as the gradual construction of graph visualization. A progressive graph visualization is introduced to monitor past inquiries, and bridge the current query with the exploration history and next-step recommendations. We demonstrate the effectiveness of our system via use cases and expert interviews.

Geospatial data is ubiquitous in today’s world, playing a crucial role in various aspects of modern life. As businesses increasingly establish and expand their online presence, they leverage location-based insights to better understand customer behavior and optimize services. Simultaneously, the proliferation of GPS-enabled devices enables users to continuously share their locations and activities online, contributing to an ever-growing pool of geospatial information. Social networks and web services, in particular, heavily rely on this data, integrating it into their platforms to provide more personalized and relevant experiences. This surge in geospatial data generation necessitates the development of novel algorithms to mine meaningful patterns and enhancing user experiences by delivering accurate information at the right place and time. This dissertation presents a pipeline of novel algorithms, spanning from the integration of geospatial data, to its effective application in real-world use cases. Specifically it introduces (1) an algorithm for deduplicating and consolidating geospatial data from multiple sources, a key step in creating a comprehensive geospatial database; it studies (2) the problem of Geospatial Knowledge Graph (GeoKG) construction, and presents an algorithm to automatically mine fine-grained spatial relationships between entities from a database, within an open-world setting; it presents (3) a framework for self-supervised pre-training of geospatial foundation models, capable of generating task-agnostic representations that are readily applicable to a wide range of downstream tasks. Finally, (4) this thesis explores the fine-tuning of pre-trained large language models (LLMs) on city-specific data, to build urban virtual assistants, capable of delivering accurate geospatial recommendations in a conversational manner while minimizing hallucinations. The study on geospatial Entity Resolution (ER) proposes a state-of-the-art approach to link records from different sources, that refer to the same real-world geospatial entity. This is a crucial step for deduplicating merged data. The algorithm, called Geo-ER, leverages textual, spatial, and contextual information, making a step forward towards automatic geospatial data integration. A novel neighborhood embedding component is designed, using Graph Attention Networks, to produce a more contextualized representation of an entity. Extensive experiments, on eight real-world datasets, are performed to demonstrate the advantages of Geo-ER compared to existing state-of-the-art solution for geospatial and traditional ER. Cross-city validation experiments showcase the algorithm's robustness in scenarios where training data for a specific city is unavailable. Subsequently, this thesis presents GTMiner, a novel framework for automatically constructing a Geospatial Knowledge Graph (GeoKG) from tabular data, derived from the merging and consolidation of multiple data sources. The architecture for geospatial relation prediction is an open-world Knowledge Graph Completion solution, consisting of a pre-trained language model, a geospatial encoder, and a Geo-Textual interaction component, which jointly model the textual and geospatial characteristics of an entity. GTMiner further includes a refinement module, specifically designed to improve both coverage and correctness of a GeoKG. The study further introduces four real-world datasets, collected from publicly available sources, and carries out extensive experiments to evaluate the performance of each of the system’s components and perform ablation studies. A study on general-purpose representation learning in the geospatial domain introduces CityFM, a new framework that leverages mutual information maximization and incorporates Nodes, Ways, and Relations into its learning frame-work to train geospatial Pre-trained Foundation Models (PFMs) for a selected area of interest. CityFM-trained models are task-agnostic and multimodal, capable of integrating an entity’s textual, visual, and spatial attributes. The study conducts experiments on road, building, and region-level downstream tasks to demonstrate the effectiveness of the embeddings produced by the foundation models in real-world applications, compared to task-specific algorithms. The last study in this dissertation presents LAMP, a virtual assistant for urban applications. It introduces a novel strategy to generate synthetic conversational data about geospatial entities, to fine-tune a pre-trained LLM. The generated data enables the language model to learn and memorize thousands of places within a city, while also equipping it with spatial (proximity) awareness specific to the urban area of interest.

The advent of Large Language Models (LLMs), such as Chat-GPT and GPT-4, has revolutionized natural language processing, opening avenues for advanced textual understanding.This study explores the application of LLMs in developing Knowledge graphs from textual data.Knowledge graphs offer a structured representation of information, facilitating enhanced comprehension and utilization of unstructured text.We intend to construct Knowledge graphs that capture relationships and entities within diverse textual datasets by harnessing LLMs' contextual understanding and language generation capabilities.The primary goal is to explore and understand how well LLMs can identify and extract relevant entities and relationships from textual data using prompt engineering while contributing to structured knowledge representation.

Drug repurposing (DR) offers a promising alternative to the high cost and low success rate of traditional drug development, especially for complex diseases like Alzheimer’s disease (AD). This study addressed DR for AD from three key angles: (1) demonstrating how disease-specific knowledge graphs can improve DR performance, (2) evaluating the role of large language models (LLMs) in enhancing the usability and efficiency of these graphs, and (3) assessing whether Graph-of-Thoughts (GoT)-enhanced LLMs, when integrated with AD knowledge graphs, can outperform traditional machine learning and LLM-based approaches. We tested five distinct DR strategies (DR1–DR5) for AD: DR1, a machine learning method using TxGNN; DR2, a machine learning model leveraging the Alzheimer’s KnowledgeBase (AlzKB); DR3, an LLM-based chatbot built on AlzKB; DR4, our ESCARGOT framework combining GoT-enhanced LLMs with AlzKB; and DR5, a general reasoning-driven LLM approach. Results showed that AlzKB significantly improved DR outcomes. ESCARGOT further enhanced performance while reducing the need for coding or advanced expertise in knowledge graph analysis. Because the architecture of AlzKB is easily adaptable to other diseases and ESCARGOT can integrate with various knowledge graph platforms, this framework offers a broadly applicable, innovative tool for accelerating drug discovery through repurposing.Supplementary InformationThe online version contains supplementary material available at 10.1186/s13040-025-00466-5.

As large language models (LLMs) continue to grow in size, their abilities to tackle complex tasks have significantly improved. However, issues such as hallucination and the lack of up-to-date knowledge largely remain unresolved. Knowledge graphs (KGs), which serve as symbolic representations of real-world knowledge, offer a reliable source for enhancing reasoning. Integrating KG retrieval into LLMs can therefore strengthen their reasoning by providing dependable knowledge. Nevertheless, due to limited understanding of the underlying knowledge graph, LLMs may struggle with queries that have multiple interpretations. Additionally, the incompleteness and noise within knowledge graphs may result in retrieval failures. To address these challenges, we propose an embedding-based retrieval reasoning framework EMBRAG. In this approach, the model first generates multiple logical rules grounded in knowledge graphs based on the input query. These rules are then applied to reasoning in the embedding space, guided by the knowledge graph, ensuring more robust and accurate reasoning. A reranker model further interprets these rules and refines the results. Extensive experiments on two benchmark KGQA datasets demonstrate that our approach achieves the new state-of-the-art performance in KG reasoning tasks.