Document GraphRAG: Knowledge Graph Enhanced Retrieval Augmented Generation for Document Question Answering Within the Manufacturing Domain

In this paper, we detail our submission to the 7th year BioASQ competition. We present our approach for Task-7b, Phase B, Exact Answering Task. These Question Answering (QA) tasks include Factoid, Yes/No, List Type Question answering. Our system is based on a contextual word embedding model. We have used a Bidirectional Encoder Representations from Transformers (BERT) based system, fined tuned for biomedical question answering task using BioBERT. In the third test batch set, our system achieved the highest ‘MRR’ score for Factoid Question Answering task. Also, for List type question answering task our system achieved the highest recall score in the fourth test batch set. Along with our detailed approach, we present the results for our submissions, and also highlight identified downsides for our current approach and ways to improve them in our future experiments.

Knowledge graph question answering is an important research direction for question answering tasks. In recent years, there has been an increasing amount of research on knowledge graph question answering, but temporal knowledge graph question answering is still a relatively unexplored area. In question answering tasks, many natural language questions have explicit or implicit temporal constraints, and most existing research methods lack the temporal awareness to deal with complex temporal questions. To address these challenges, this paper proposes a question answering model based on temporal knowledge graph embedding (TKGETQA). The model uses TKG embeddings to root a question in the entities, relations and time horizons it references, and perceives temporal information from the question to improve the accuracy of answer prediction. Experiments on the dataset CronQuestions in this paper show that the TKGETQA model exhibits better results compared to existing temporal knowledge graph question answering approaches.

While conversing with chatbots, humans typically tend to ask many questions, a significant portion of which can be answered by referring to large-scale knowledge graphs (KG). While Question Answering (QA) and dialog systems have been studied independently, there is a need to study them closely to evaluate such real-world scenarios faced by bots involving both these tasks. Towards this end, we introduce the task of Complex Sequential QA which combines the two tasks of (i) answering factual questions through complex inferencing over a realistic-sized KG of millions of entities, and (ii) learning to converse through a series of coherently linked QA pairs. Through a labor intensive semi-automatic process, involving in-house and crowdsourced workers, we created a dataset containing around 200K dialogs with a total of 1.6M turns. Further, unlike existing large scale QA datasets which contain simple questions that can be answered from a single tuple, the questions in our dialogs require a larger subgraph of the KG. Specifically, our dataset has questions which require logical, quantitative, and comparative reasoning as well as their combinations. This calls for models which can: (i) parse complex natural language questions, (ii) use conversation context to resolve coreferences and ellipsis in utterances, (iii) ask for clarifications for ambiguous queries, and finally (iv) retrieve relevant subgraphs of the KG to answer such questions. However, our experiments with a combination of state of the art dialog and QA models show that they clearly do not achieve the above objectives and are inadequate for dealing with such complex real world settings. We believe that this new dataset coupled with the limitations of existing models as reported in this paper should encourage further research in Complex Sequential QA.

No one is perfect: Analysing the performance of question answering components over the DBpedia knowledge graph

Retrieval-augmented generation (RAG) offers an effective approach for addressing question answering (QA) tasks. However, the imperfections of the retrievers in RAG models often result in the retrieval of irrelevant information, which could introduce noise and degrade the performance, especially when handling multi-hop questions that require multiple steps of reasoning. To enhance the multi-hop reasoning ability of RAG models, we propose TRACE1. TRACE constructs knowledge-grounded reasoning chains, which are a series of logically connected knowledge triples, to identify and integrate supporting evidence from the retrieved documents for answering questions. Specifically, TRACE employs a KG Generator to create a knowledge graph (KG) from the retrieved documents, and then uses a novel Autoregressive Reasoning Chain Constructor to build reasoning chains. Experimental results on three multi-hop QA datasets show that TRACE achieves an average performance improvement of up to 14.03% compared to using all the retrieved documents. Moreover, the results indicate that using reasoning chains as context, rather than the entire documents, is often sufficient to correctly answer questions.

Large Language Models (LLMs) face significant challenges when attempting to utilize knowledge not encoded during pre-training. In-context learning (ICL) addresses this limitation by embedding relevant information directly into prompts, enabling LLMs to leverage external knowledge without updating their parameters. Recent advances have explored integrating knowledge graph (KG) information into prompts, taking advantage of the structured and factual representations of entities and relationships that KGs provide. A common approach involves identifying key concepts in a task, grounding them to KG nodes, extracting ego-subgraphs centered on these concepts, and incorporating them into prompts. However, not all subgraphs are equally relevant to the task, and selecting appropriate knowledge remains a critical challenge, as irrelevant or noisy subgraphs can reduce model accuracy. This study investigates how KG-based information related to task-specific concepts influences LLM performance on multiple-choice question answering (MCQA) tasks. We implement a four-stage pipeline that (1) identifies key concepts from questions and grounds them in the KG, (2) extracts individual ego-subgraphs for each concept, (3) integrates these subgraphs into prompts, and (4) evaluates their impact on LLM reasoning via probability scoring. Our analysis highlights which concept-specific subgraphs enhance performance, which introduce misleading information, and which have neutral effects. We show that carefully selected KG subgraphs can substantially outperform others in semantic relevance. Furthermore, we examine how the size and connectivity of selected KG subgraphs influence model performance. Overall, this work deepens understanding of KG-based knowledge selection in ICL and informs the design of more effective, targeted prompting strategies. This project's source code is publicly available at https://github.com/maryam-ghanbari/InformativenessOfKGs.

In this paper, we present a model which incorporates biomedical knowledge graph, graph embedding and deep learning methods for literature-based discovery. Firstly, the relations between entities are extracted from biomedical abstracts and then a knowledge graph is constructed by using these obtained relations. Secondly, the graph embedding technologies are applied to convert the entities and relations in the knowledge graph into a low-dimensional vector space. Thirdly, a bidirectional Long Short-Term Memory network is trained based on the entity associations represented by the pre-trained graph embeddings. Finally, the learned model is used for open and closed literature-based discovery tasks. The experimental results show that our method could not only effectively discover hidden associations between entities, but also reveal the corresponding mechanism of interactions. It suggests that incorporating knowledge graph and deep learning methods is an effective way for capturing the underlying complex associations between entities hidden in the literature.

DTN: Deep triple network for topic specific fake news detection

A review of recommender systems based on knowledge graph embedding

In multi-hop question answering (MHQA) tasks, existing methods typically integrate multiple reasoning paths from knowledge graphs (KGs) and chains of thought (CoTs). Early KG-enhanced methods primarily focus on obtaining relevant knowledge but fail to consider the multi-dimensional quality of reasoning paths. Subsequent works filter paths but treat all retained paths as equally important without further differentiation. Although some recent works attempt to rank paths by quality, they only provide a relative order without quantifying the actual quality differences between paths. To address these limitations, we propose a Multi-dimensional Quality-aware Path Fusion (MQPF) framework. MQPF introduces a multi-dimensional evaluation mechanism that quantifies path quality from semantic, structural, and outcome-based dimensions. Based on the overall scores, MQPF first filters out low-quality paths to reduce noise and then assigns adaptive weights to the remaining paths according to their scores. This method effectively removes unreliable information and enhances the utilization of trustworthy information during reasoning. Experiments show that MQPF performs comparably to baselines on multiple datasets. Moreover, as a model-agnostic module, it can be used as a plug-and-play module to enhance the performance of existing multi-path reasoning methods.

We introduce SPARTA, a novel neural retrieval method that shows great promise in performance, generalization, and interpretability for open-domain question answering. Unlike many neural ranking methods that use dense vector nearest neighbor search, SPARTA learns a sparse representation that can be efficiently implemented as an Inverted Index. The resulting representation enables scalable neural retrieval that does not require expensive approximate vector search and leads to better performance than its dense counterpart. We validated our approaches on 4 open-domain question answering (OpenQA) tasks and 11 retrieval question answering (ReQA) tasks. SPARTA achieves new state-of-the-art results across a variety of open-domain question answering tasks in both English and Chinese datasets, including open SQuAD, CMRC and etc. Analysis also confirms that the proposed method creates human interpretable representation and allows flexible control over the trade-off between performance and efficiency.

Learning structured embeddings of knowledge graphs with generative adversarial framework

It is challenging to generate high quality answers for healthcare queries in online platforms. Recent studies proposed deep models for healthcare question answering (HQA) tasks. However, these models have not been thoroughly compared, and they were only tested on self-created datasets. This paper demonstrates a novel system, denoted by HQADeepHelper, to facilitate the learning and practicing of deep models for HQA. We have implemented a wide spectrum of state-of-the-art deep models for HQA retrieval. Users can upload self-collected HQA datasets and knowledge graphs, and do simple configurations by selecting datasets, knowledge graphs, neural network models, and evaluation metrics. Based on user’s configuration specified, the system can automatically train and test the model, conduct extensive experimental evaluation of the models selected, and report comprehensive findings. The reports provide new insights about the strengths and weaknesses of deep models that can guide practitioners to select appropriate models for various scenarios. Moreover, users can download the datasets, knowledge graphs, experimental reports and source codes of neural network models for their own practice and evaluations further.

Effective and efficient provision and reuse of knowledge across a knowledge network in the global value chain still faces two challenges, namely the interpretability of model and the efficacy of construction. This research aims to address these challenges by proposing a novel representation model for design knowledge. First, a knowledge representation model based on the integration of the cognitive process theory and the Requirement-Function-Behavior-Structure model (C-RFBS model) is proposed to incorporate key elements from the cognitive process of designers to capture the rationale of deliberation and the context of decision-making, which the knowledge records created become more interpretable. Second, knowledge graph is employed to improve the productivity of knowledge records creation, storage and exploration. On this basis, we describe the creation of knowledge records using the C-RFBS model as well as the computational framework and methods for storing knowledge using knowledge graph. The proposed model and methods are implemented in a knowledge retrieval system on which we have conducted a fork design case study to evaluate and demonstrate the models and methods. As shown in the evaluation, the proposed model can effectively support knowledge elicitation and achieved improved performance in terms of knowledge retrieval through incorporating knowledge graph.

Knowledge-aware fine-grained attention networks with refined knowledge graph embedding for personalized recommendation