- Research Article
189
- 10.1016/j.cmpb.2009.10.003
Biomedical question answering: A survey
- Nov 13, 2009
- Computer Methods and Programs in Biomedicine
- Sofia J Athenikos + 1 more +1
Biomedical question answering: A survey
BackgroundOne of the challenges in large-scale information retrieval (IR) is developing fine-grained and domain-specific methods to answer natural language questions. Despite the availability of numerous sources and datasets for answer retrieval, Question Answering (QA) remains a challenging problem due to the difficulty of the question understanding and answer extraction tasks. One of the promising tracks investigated in QA is mapping new questions to formerly answered questions that are “similar”.ResultsWe propose a novel QA approach based on Recognizing Question Entailment (RQE) and we describe the QA system and resources that we built and evaluated on real medical questions. First, we compare logistic regression and deep learning methods for RQE using different kinds of datasets including textual inference, question similarity, and entailment in both the open and clinical domains. Second, we combine IR models with the best RQE method to select entailed questions and rank the retrieved answers. To study the end-to-end QA approach, we built the MedQuAD collection of 47,457 question-answer pairs from trusted medical sources which we introduce and share in the scope of this paper. Following the evaluation process used in TREC 2017 LiveQA, we find that our approach exceeds the best results of the medical task with a 29.8% increase over the best official score.ConclusionsThe evaluation results support the relevance of question entailment for QA and highlight the effectiveness of combining IR and RQE for future QA efforts. Our findings also show that relying on a restricted set of reliable answer sources can bring a substantial improvement in medical QA.
Loading PDF
Biomedical question answering: A survey
Biomedical question answering: A survey
Epidemic Question Answering: question generation and entailment for Answer Nugget discovery.
The rapidly growing body of communications during the COVID-19 pandemic posed a challenge to information seekers, who struggled to find answers to their specific and changing information needs. We designed a Question Answering (QA) system capable of answering ad-hoc questions about the COVID-19 disease, its causal virus SARS-CoV-2, and the recommended response to the pandemic. The QA system incorporates, in addition to relevance models, automatic generation of questions from relevant sentences. We relied on entailment between questions for (1) pinpointing answers and (2) selecting novel answers early in the list of its results. The QA system produced state-of-the-art results when processing questions asked by experts (eg, researchers, scientists, or clinicians) and competitive results when processing questions asked by consumers of health information. Although state-of-the-art models for question generation and question entailment were used, more than half of the answers were missed, due to the limitations of the relevance models employed. Although question entailment enabled by automatic question generation is the cornerstone of our QA system's architecture, question entailment did not prove to always be reliable or sufficient in ranking the answers. Question entailment should be enhanced with additional inferential capabilities. The QA system presented in this article produced state-of-the-art results processing expert questions and competitive results processing consumer questions. Improvements should be considered by using better relevance models and enhanced inference methods. Moreover, experts and consumers have different answer expectations, which should be accounted for in future QA development.
Read moreANU-CSIRO at MEDIQA 2019: Question Answering Using Deep Contextual Knowledge
We report on our system for textual inference and question entailment in the medical domain for the ACL BioNLP 2019 Shared Task, MEDIQA. Textual inference is the task of finding the semantic relationships between pairs of text. Question entailment involves identifying pairs of questions which have similar semantic content. To improve upon medical natural language inference and question entailment approaches to further medical question answering, we propose a system that incorporates open-domain and biomedical domain approaches to improve semantic understanding and ambiguity resolution. Our models achieve 80% accuracy on medical natural language inference (6.5% absolute improvement over the original baseline), 48.9% accuracy on recognising medical question entailment, 0.248 Spearman’s rho for question answering ranking and 68.6% accuracy for question answering classification.
Read moreLearning to select the correct answer in multi-stream question answering
Learning to select the correct answer in multi-stream question answering
A Pre-trained Language Model for Medical Question Answering Based on Domain Adaption
With the successful application of question answering (QA) in human-computer interaction scenarios such as chatbots and search engines, medical question answering (QA) systems have gradually attracted widespread attention, because it can not only help professionals make decisions efficiently, but also supply non-professional people advice when they are seeking useful information. However, due to the professionalism of domain knowledge, it is still hard for existing medical question answering systems to understand professional domain knowledge of medicine, which makes question answering systems unable to generate fluent and accurate answers. The goal of this paper is to train the language model on the basis of pre-training. With better language models, we can get better medical question answering models. Through the combination of DAP and TAP, the model can understand the knowledge of the medical domain and task, which helps question answering models generate smooth and accurate answers and achieve good results.KeywordsQuestion answeringDAPTAPPretraining
Read moreOverview of the MEDIQA 2019 Shared Task on Textual Inference, Question Entailment and Question Answering
This paper presents the MEDIQA 2019 shared task organized at the ACL-BioNLP workshop. The shared task is motivated by a need to develop relevant methods, techniques and gold standards for inference and entailment in the medical domain, and their application to improve domain specific information retrieval and question answering systems. MEDIQA 2019 includes three tasks: Natural Language Inference (NLI), Recognizing Question Entailment (RQE), and Question Answering (QA) in the medical domain. 72 teams participated in the challenge, achieving an accuracy of 98% in the NLI task, 74.9% in the RQE task, and 78.3% in the QA task. In this paper, we describe the tasks, the datasets, and the participants’ approaches and results. We hope that this shared task will attract further research efforts in textual inference, question entailment, and question answering in the medical domain.
Read moreConstruction of a Knowledge Graph-based Medical Question Answer System
Artificial intelligence is widely used in the field of medical question answering. Some existing medical domain knowledge graphs have problems such as few categories of disease entities covered and small coverage of question answering. In this paper, a medical domain knowledge graph containing nearly 10,000 diseases is constructed, and a medical domain question answering system is implemented based on this knowledge graph using the Bert model. Experiment results show that the accuracy and recall of the system reach 0.96 and 0.93 respectively, with an F1 value of 0.945, which can effectively answer users' questions about the diagnosis of symptoms and medication recommendation of vast majority diseases and save medical resources to a certain extent.
Read moreMedical question answering: A comprehensive multimodal and LLM-driven survey.
Medical question answering: A comprehensive multimodal and LLM-driven survey.
A survey on textual entailment based question answering
A survey on textual entailment based question answering
A Survey Of Challenges And Resolutions Of Mining Question-Answer Pairs From Internet Forum
Internet forum is a web community that brings people in different geographical locations together. Members of the forum exchange ideas and expertise and as a result generate huge amount of content on different topics on daily basis. A good percentage of human generated content of Internet forums have been found to be question-answer (QA) pairs. These QA pairs are useful for automating question answering system. Mining these QA pairs has become a hot issue in the research community. Effective mining of the QA pairs is being hindered by a number of factors. Lexical chasm that renders some Information Retrieval (IR) techniques less effective, casual language that creates noisy data; multiple authors that bring about unfocused topics are some of the issues that need to be addressed. In this paper, an extensive overview of the strategies and findings relevant to these three challenges are addressed. The survey revealed that researchers are adopting non-lexical features as against lexical to resolve the issue of data sparseness. Noise level is mostly controlled using conventional dictionary rather than using domain-specific dictionary.
Read moreThai Question-Answering System Using Similarity Search and LLM
A question-answering (QA) system is essential to an organization where numerous QA pairs respond to customer queries. Choosing the right pair corresponding to the query is a complex task. Although the QA system from a commercial product like ChatGPT provides an excellent solution, it is costly, and the fine-tuned Large Language Model (LLM) cannot be downloaded for private use at the local site. In addition, the cost of using such LLM may significantly increase when the number of users grows. We propose a Thai QA system that can swiftly respond and correctly match the user query to the reference answer in the QA dataset. The proposed system encodes both QA pairs and a query into individual embeddings and finds a couple of QA pairs that are most related to the query by using the fast similarity search called Faiss (Facebook AI Similarity Search.) Afterward, the relevant QA pairs and the query are fed to the fine-tuned LLM (WangchanBERTa - pretraining multilingual transformer-based) to choose the single best match QA pair. The fine-tuned WangchanBERTa can retrieve the correct answer and respond to the query naturally. The experiment conducted on the Thai Wiki QA dataset indicates the superior ROUGE values, precision, recall, F1-score, and runtime of the proposed system against other strategies.
Read moreDeveloping ChatGPT for biology and medicine: a complete review of biomedical question answering.
ChatGPT explores a strategic blueprint of question answering (QA) to deliver medical diagnoses, treatment recommendations, and other healthcare support. This is achieved through the increasing incorporation of medical domain data via natural language processing (NLP) and multimodal paradigms. By transitioning the distribution of text, images, videos, and other modalities from the general domain to the medical domain, these techniques have accelerated the progress of medical domain question answering (MDQA). They bridge the gap between human natural language and sophisticated medical domain knowledge or expert-provided manual annotations, handling large-scale, diverse, unbalanced, or even unlabeled data analysis scenarios in medical contexts. Central to our focus is the utilization of language models and multimodal paradigms for medical question answering, aiming to guide the research community in selecting appropriate mechanisms for their specific medical research requirements. Specialized tasks such as unimodal-related question answering, reading comprehension, reasoning, diagnosis, relation extraction, probability modeling, and others, as well as multimodal-related tasks like vision question answering, image captioning, cross-modal retrieval, report summarization, and generation, are discussed in detail. Each section delves into the intricate specifics of the respective method under consideration. This paper highlights the structures and advancements of medical domain explorations against general domain methods, emphasizing their applications across different tasks and datasets. It also outlines current challenges and opportunities for future medical domain research, paving the way for continued innovation and application in this rapidly evolving field. This comprehensive review serves not only as an academic resource but also delineates the course for future probes and utilization in the field of medical question answering.
Read moreWhat Would it Take to get Biomedical QA Systems into Practice?
Medical question answering (QA) systems have the potential to answer clinicians' uncertainties about treatment and diagnosis on-demand, informed by the latest evidence. However, despite the significant progress in general QA made by the NLP community, medical QA systems are still not widely used in clinical environments. One likely reason for this is that clinicians may not readily trust QA system outputs, in part because transparency, trustworthiness, and provenance have not been key considerations in the design of such models. In this paper we discuss a set of criteria that, if met, we argue would likely increase the utility of biomedical QA systems, which may in turn lead to adoption of such systems in practice. We assess existing models, tasks, and datasets with respect to these criteria, highlighting shortcomings of previously proposed approaches and pointing toward what might be more usable QA systems.
Read moreRetrieval-Augmented Generation for Medical Question Answering on a Heart Failure Dataset: Performance Analysis.
The integration of retrieval-augmented generation (RAG) systems into the domain of medical question-answering (QA) presents a significant opportunity to enhance the effectiveness and accuracy of clinical support systems. This study aimed to explore the design choices within the RAG framework and the use of large language model (LLM) classifiers to optimize medical QA systems, enhancing response quality for patient and caregiver queries of varying risk levels. In total, we curated a dataset of 109 patient and caregiver questions related to heart failure (HF)-categorized into answerable (direct, fact-based queries), helpful deferral (general guidance or lifestyle advisory queries), and nonanswerable (out-of-scope or high-risk and medical intervention queries) types-along with relevant documents and a target answer for each question from the website The Heart Hub. Applying a system architecture leveraging RAG with a structured query taxonomy and robust classification mechanisms, this paper provided an empirical assessment for medical QA on a HF dataset and introduced a QA system pipeline design, providing a foundation for extended application across various medical fields. Specifically, we evaluated design choices in the initial retrieval stage of RAG and their impact on performance. We assessed final answer quality from the generation stage using popular passage scoring methods for QA, such as Recall-Oriented Understudy for Gisting Evaluation (ROUGE), BERTScore, and Intersection over Union score. The pipeline first uses an LLM-based classifier, achieving 65% accuracy for answerable and helpful deferral queries and 100% accuracy for identifying nonanswerable queries. In information retrieval, the BioMedical Contrastive Pre-trained Transformers (MedCPT) cross encoder performed best as a dense retrieval method, delivering an average of 93% recall @ 7 through ranked relevance scores to obtain the top documents with recall @ k denoting recall computed over the top-k retrieved items. For further retrieving snippets from such documents, its average performance was 72.5% for sentence-level snippets and 83% for paragraph-level snippets. A second LLM-based classifier, used to refine the generated responses, resulted in an overall reduction in ROUGE-1 recall by 13% and Bidirectional Encoder Representations from Transformers (BERT) precision by 11%. However, Intersection over Union scores, or the overlap between "gold answers" and system answers, increased by 24%, demonstrating enhanced alignment with ground truth responses. This also indicates the system's improved ability to generate concise and accurate medical responses. The implementation of a structured RAG framework paired with LLM classifiers for medical QA introduces a promising avenue for enhancing clinical decision support systems. By systematically analyzing the impact of query taxonomy, retrieval configurations, and response strategies, this approach clarifies the relative importance of each component within the medical RAG system using a HF dataset. Our findings provide actionable guidance on optimal design choices for maximizing retrieval and response accuracy; thus, informing the development of robust, scalable medical QA systems.
Read moreStatMetaQA: A dataset for closed domain question answering in Indonesian statistical metadata
StatMetaQA: A dataset for closed domain question answering in Indonesian statistical metadata