Knowledge Base Question Answering With a Matching-Aggregation Model and Question-Specific Contextual Relations

PurposeThe purpose of this paper is to build a better question answering (QA) system that can furnish more improved retrieval of answers related to COVID-19 queries from the COVID-19 open research data set (CORD-19). As CORD-19 has an up-to-date collection of coronavirus literature, text mining approaches can be successfully used to retrieve answers pertaining to all coronavirus-related questions. The existing a lite BERT for self-supervised learning of language representations (ALBERT) model is finetuned for retrieving all COVID relevant information to scientific questions posed by the medical community and to highlight the context related to the COVID-19 query.Design/methodology/approachThis study presents a finetuned ALBERT-based QA system in association with Best Match25 (Okapi BM25) ranking function and its variant BM25L for context retrieval and provided high scores in benchmark data sets such as SQuAD for answers related to COVID-19 questions. In this context, this paper has built a QA system, pre-trained on SQuAD and finetuned it on CORD-19 data to retrieve answers related to COVID-19 questions by extracting semantically relevant information related to the question.FindingsBM25L is found to be more effective in retrieval compared to Okapi BM25. Hence, finetuned ALBERT when extended to the CORD-19 data set provided accurate results.Originality/valueThe finetuned ALBERT QA system was developed and tested for the first time on the CORD-19 data set to extract context and highlight the span of the answer for more clarity to the user.

Open-domain question answering is an emerging field of natural language processing. Application areas of question answering (QA) systems are increasing with the availability of new datasets, more powerful machines and easy access to voice-enabled devices that interact with humans in their daily lives. Users can get into interactions with QA systems without the need to be limited by any domain. However, there is still need for QA systems that can answer questions in a specific field such as international criminal law, violence of human rights or bill of rights. The recent approaches on open-domain question answering made it possible to extend open-domain QA systems towards specific fields. This paper presents Quill-QA, the new feature of the Quill Platform. Quill Platform is a web platform providing set of tools that digitize and visualize negotiated texts, any text in the form of formal negotiations. Quill-QA is based on Facebook's DrQA system and as a result it adopts a flexible Ranker-Reader architecture. Quill-QA answers questions addressed on the samples of negotiated texts such as U.S. Constitutional Convention of 1787, Utah State Constitutional Convention of 1895 and European Union Act 2017. Without limiting the scope of the input questions to a finite set of templates, Quill-QA is the first QA system that focuses on negotiated texts.

Given a user question, the goal of a Question Answering (QA) system is to retrieve answers rather than full documents or even best-matching passages, as most Information Retrieval systems currently do. In this paper, we present BRUJA, a QA system for the management of multilingual collections. BRUJ rkstions (English, Spanish and French). The BRUJA architecture is not formed with three monolingual QA systems but instead uses English as Interlingua to make usual QA tasks such as question classifications and answer extractions. In addition, BRUJA uses Cross Language Information Retrieval (CLIR) techniques to retrieve relevant documents from a multilingual collection. On the one hand, we have more documents to find answers from but on the other hand, we are introducing noise into the system because of translations to the Interlingua (English) and the CLIR module. The question is whether the difficulty of managing three languages is worth it or whether a monolingual QA system delivers better results. We report on in-depth experimentation and demonstrate that our multilingual QA system gets better results than its monolingual counterpart whenever it uses good translation resources and, especially, CLIR techniques that are state-of-the-art.

This paper presents a study of the negative effect of Machine Translation (MT) on the precision of Cross---Lingual Question Answering (CL---QA). For this research, a English---Spanish Question Answering (QA) system is used. Also, the sets of 200 official questions from CLEF 2004 and 2006 are used. The CL experimental evaluation using MT reveals that the precision of the system drops around 30% with regard to the monolingual Spanish task. Our main contribution consists on a taxonomy of the identified errors caused by using MT and how the errors can be overcome by using our proposals. An experimental evaluation proves that our approach performs better than MT tools, at the same time contributing to this CL---QA system being ranked first at English---Spanish QA CLEF 2006.

Number of people having expertise in a certain domain is less than people who need information in that domain. In this situation, an automatic question answering (QA) system is necessary. Observing available manual QA sites on internet, the real world question that people usually ask have different expected answer type (EAT) compared to a common automatic QA. Addressing a case study of a religion domain which makes it a closed domain QA, we proposed the EAT into 6 types: LAW, DEFINITION, COMPARISON, METHOD, TIME and PERSON. Different with common QA approach, we built the QA system using case based approach which consists of two main components: Question Analyzer and Case Retriever. Related with the case based reasoning (CBR) framework, these two main components act as the Retrieve and Reuse process while the Revise and Retain process is handle by Case Retainer component. The QA system was built using available Indonesian Natural Language Processing (NLP) tools and FreeCBR as the CBR library. The experiments were done to calculate the accuracy and testing the system with unknown case. By using 77 cases collected from internet with assumption that all answers are available, the experiments achieved 97% accuracy. And by using 10 test cases for the unknown case, the similarity score calculated by the system showed that the test questions have no answer in the available case base.

The goal of the question-answering (QA) system is to understand the questions from users and return their accurate answers. In the medical field, the question-answering system amis to understand patients' questions and return the correct answers. The existed knowledge base question-answering (KB-QA) systems mainly rely on hand-crafted features and ignore structure information of knowledge base which accordingly lead to the answers with low accuracy. In this paper, a novel KB-QA model is put forward based on knowledge representation and recurrent convolutional neural network. This model has three parts, candidate answers generation, entity relationships extraction and knowledge representation learning based on knowledge base. In addition, an algorithm is also developed to compute the scores of linking candidate answers and knowledge base. Experimental results show that the presented model achieves better performance compared with the baseline systems.

Information can also be a means of learning for humans. Including information about history because history can be a means of learning for the younger generation to appreciate the nation's culture and build national identity. In the past, the Sumedang Larang kingdom was one of the many kingdoms in West Java, Indonesia, that could be used as much information as a lesson. Technological developments make more and more information available for study. We need the proper means to find the information we need. This study aims to build a Question Answering (QA) system to create a means for the younger generation to be more familiar with the history of the kingdom in the past. The QA system offers an information retrieval system that is easy to access and can immediately provide the answers we need. This QA system was built using ontology as a knowledge base and cosine similarity to determine the similarity between user questions and the dataset. The QA system that has been built is tested by providing a set of questions so that the system's performance can be measured, and the results of system testing get a precision value of 70% and a recall value of 90%.

Previous works on evaluating the performance of Question Answering (QA) systems are focused on the evaluation of the precision. In this paper, we developed a mathematic procedure in order to explore new evaluation measures in QA systems considering the answer time. Also, we carried out an exercise for the evaluation of QA systems within a time constraint in the CLEF-2006 campaign, using the proposed measures. The main conclusion is that the evaluation of QA systems in realtime can be a new scenario for the evaluation of QA systems.

Watson, the winner of the Jeopardy! challenge, is a state-of-the-art open-domain Question Answering system that tackles the fundamental issue of answer typing by using a novel type coercion (TyCor) framework, where candidate answers are initially produced without considering type information, and subsequent stages check whether the candidate can be coerced into the expected answer type. In this paper, we provide a high-level overview of the TyCor framework and discuss how it is integrated in Watson, focusing on and evaluating three TyCor components that leverage the community built semi-structured and structured knowledge resources – DBpedia (in conjunction with the YAGO ontology), Wikipedia Categories and Lists. These resources complement each other well in terms of precision and granularity of type information, and through links to Wikipedia, provide coverage for a large set of instances.

In recent years, medical Question Answering (QA) systems have gained a lot of attentions, since they can not only help professionals make quick decisions, but also give ordinary people advice when they seek for helpful information. However, the interpretability and accuracy problems have plagued QA systems for a long time. In this paper, we propose a QA system, DSQA, based on knowledge graph for answering domain-specific medical questions. The data we use are from Electronic Medical Records (EMRs) that are complex and heterogeneous with varied qualities. We introduce doctor-in-the-loop mechanism to design the knowledge graph, which improves the interpretability and the accuracy of our QA system. For every natural-language question, we generate the answer with a hybrid of the traditional deep learning model and the reasoning on the knowledge graph. Finally, the answer is returned to the user in the format of a subgraph with corresponding natural-language sentences. Our system shows high interpretability and gives excellent performance according to the experiment results.

Question answering systems in general use external knowledge sources for extracting answers. Domain specific question answering systems require pre-constructed knowledge sources like domain ontology. A major challenge in knowledge-based question answering system development is building a huge knowledge base with the objective and correct factual knowledge in the preferred domain. The process of collecting useful knowledge from various sources and maintaining in a knowledge repository is a useful process for providing required answer on demand with greater accuracy and efficiency. In this paper an experimental framework has been proposed for concept-relational ontology-based question answering process. The question answering framework proposed in this paper includes two subsystems: 1) a dynamic concept relational ontology construction module, is capable to extract new concepts from the web and incorporate the extracted knowledge into the concept relational ontology knowledge base; 2) the answer extraction module formulates the query string from the natural language question according to the expected answer and retrieves the information from the ontology for answer formation. Furthermore, a novel statistical validation framework for evaluating the prototype implementation of the proposed question answering system is introduced.

Datasets that boosted state-of-the-art solutions for Question Answering (QA) systems prove that it is possible to ask questions in natural language manner. However, users are still used to query-like systems where they type in keywords to search for answer. In this study we validate which parts of questions are essential for obtaining valid answer. In order to conclude that, we take advantage of LIME - a framework that explains prediction by local approximation. We find that grammar and natural language is disregarded by QA. State-of-the-art model can answer properly even if 'asked' only with a few words with high coefficients calculated with LIME. According to our knowledge, it is the first time that QA model is being explained by LIME.

While RDF was designed to make data easily readable by machines, it does not make data easily usable by end-users. Question Answering (QA) over Knowledge Graphs (KGs) is seen as the technology which is able to bridge this gap. It aims to build systems which are capable of extracting the answer to a user’s natural language question from an RDF dataset.In recent years, many approaches were proposed which tackle the problem of QA over KGs. Despite such efforts, it is hard and cumbersome to create a Question Answering system on top of a new RDF dataset. The main open challenge remains portability, i.e., the possibility to apply a QA algorithm easily on new and previously untested RDF datasets.In this publication, we address the problem of portability by presenting an architecture for a portable QA system. We present a novel approach called QAnswer KG, which allows the construction of on-demand QA systems over new RDF datasets. Hence, our approach addresses non-expert users in QA domain.In this paper, we provide the details of QA system generation process. We show that it is possible to build a QA system over any RDF dataset while requiring minimal investments in terms of training. We run experiments using 3 different datasets.To the best of our knowledge, we are the first to design a process for non-expert users. We enable such users to efficiently create an on-demand, scalable, multilingual, QA system on top of any RDF dataset.

In this paper, we propose an alignment-based surface pattern approach, called ABSP, which integrates semantic information into syntactic patterns for question answering (QA). ABSP uses surface patterns to extract important terms from questions, and constructs the terms’ relations from sentences in the corpus. The relations are then used to filter appropriate answer candidates. Experiments show that ABSP can achieve high accuracy and can be incorporated into other QA systems that have high coverage. It can also be used in cross-lingual QA systems. The approach is both robust and portable to other domains.

This paper presents a Retrieval-Augmented Generation (RAG) system for question answering over the set of documents of a public tender. The proof-of-concept chatbot is intended to provide support to officers participating in the evaluation process of a tender whose documents are indexed. The system implements a dual-mode retrieval strategy —supporting both semantic and lexical search— to extract relevant context and uses an instruction-following language model to generate grounded responses based strictly on the retrieved information. The system is designed to assist officers in exploring tenders, evaluation reports, and award justifications through natural language queries. In order to assess the system, the documents of a tender have been generated symtheticly. Qualitative testing demonstrated that the system provided accurate, verifiable answers, avoiding unsupported inferences. The architecture emphasizes document fidelity, interpretability, and usability for non-technical users involved in procurement oversight. Future work may extend the framework to support the identification of fraud-related patterns.