Named Entity Aware Transfer Learning for Biomedical Factoid Question Answering.

Factoid question answering (QA) has recently benefited from the development of deep learning (DL) systems. Neural network models outperform traditional approaches in domains where large datasets exist, such as SQuAD (ca. 100,000 questions) for Wikipedia articles. However, these systems have not yet been applied to QA in more specific domains, such as biomedicine, because datasets are generally too small to train a DL system from scratch. For example, the BioASQ dataset for biomedical QA comprises less then 900 factoid (single answer) and list (multiple answers) QA instances. In this work, we adapt a neural QA system trained on a large open-domain dataset (SQuAD, source) to a biomedical dataset (BioASQ, target) by employing various transfer learning techniques. Our network architecture is based on a state-of-the-art QA system, extended with biomedical word embeddings and a novel mechanism to answer list questions. In contrast to existing biomedical QA systems, our system does not rely on domain-specific ontologies, parsers or entity taggers, which are expensive to create. Despite this fact, our systems achieve state-of-the-art results on factoid questions and competitive results on list questions.

Machine Reading Comprehension (MRC) is a challenging Question - Answering (QA) task that helps the user in providing the answer to the given question. There is a lot of progress in this area due to the availability of large datasets and large pre-trained language models based on transformer architecture (BERT). Named Entity Recognition (NER) was used for neural QA systems to improve performance. However, whether NER plays a vital role in a QA system built using contextual embeddings obtained through BERT variants is not explored. To fill this gap, we investigate whether NER is helpful in improving the performance of QA systems built using BERT variants. We experimented with Squad 2.0 using SpanBERT. The Squad 2.0 dataset has both answerable and unanswerable questions. The proposed model finds the answer span if the question is answerable and, provides justification for the unanswerable questions. We perform question analysis to find the expected answer tag and then use that information to find the relevant parts of the passage in order to retrieve the answer span.

Open-domain question answering is an emerging field of natural language processing. Application areas of question answering (QA) systems are increasing with the availability of new datasets, more powerful machines and easy access to voice-enabled devices that interact with humans in their daily lives. Users can get into interactions with QA systems without the need to be limited by any domain. However, there is still need for QA systems that can answer questions in a specific field such as international criminal law, violence of human rights or bill of rights. The recent approaches on open-domain question answering made it possible to extend open-domain QA systems towards specific fields. This paper presents Quill-QA, the new feature of the Quill Platform. Quill Platform is a web platform providing set of tools that digitize and visualize negotiated texts, any text in the form of formal negotiations. Quill-QA is based on Facebook's DrQA system and as a result it adopts a flexible Ranker-Reader architecture. Quill-QA answers questions addressed on the samples of negotiated texts such as U.S. Constitutional Convention of 1787, Utah State Constitutional Convention of 1895 and European Union Act 2017. Without limiting the scope of the input questions to a finite set of templates, Quill-QA is the first QA system that focuses on negotiated texts.

Given a user question, the goal of a Question Answering (QA) system is to retrieve answers rather than full documents or even best-matching passages, as most Information Retrieval systems currently do. In this paper, we present BRUJA, a QA system for the management of multilingual collections. BRUJ rkstions (English, Spanish and French). The BRUJA architecture is not formed with three monolingual QA systems but instead uses English as Interlingua to make usual QA tasks such as question classifications and answer extractions. In addition, BRUJA uses Cross Language Information Retrieval (CLIR) techniques to retrieve relevant documents from a multilingual collection. On the one hand, we have more documents to find answers from but on the other hand, we are introducing noise into the system because of translations to the Interlingua (English) and the CLIR module. The question is whether the difficulty of managing three languages is worth it or whether a monolingual QA system delivers better results. We report on in-depth experimentation and demonstrate that our multilingual QA system gets better results than its monolingual counterpart whenever it uses good translation resources and, especially, CLIR techniques that are state-of-the-art.

Cervical cancer is one of the fastest growing global health problems and leading cause of mortality among women of developing countries. Automated Pap smear cell recognition and classification in early stage of cell development is crucial for effective disease diagnosis and immediate treatment. Thus, in this article, we proposed a novel internet of health things (IoHT)-driven deep learning framework for detection and classification of cervical cancer in Pap smear images using concept of transfer learning. Following transfer learning, convolutional neural network (CNN) was combined with different conventional machine learning techniques like K nearest neighbor, naive Bayes, logistic regression, random forest and support vector machines. In the proposed framework, feature extraction from cervical images is performed using pre-trained CNN models like InceptionV3, VGG19, SqueezeNet and ResNet50, which are fed into dense and flattened layer for normal and abnormal cervical cells classification. The performance of the proposed IoHT frameworks is evaluated using standard Pap smear Herlev dataset. The proposed approach was validated by analyzing precision, recall, F1-score, training–testing time and support parameters. The obtained results concluded that CNN pre-trained model ResNet50 achieved the higher classification rate of 97.89% with the involvement of random forest classifier for effective and reliable disease detection and classification. The minimum training time and testing time required to train model were 0.032 s and 0.006 s, respectively.

Breast cancer comprises a serious public health concern. The three primary techniques for detecting breast cancer are ultrasound, mammography, and magnetic resonance imaging (MRI). However, the existing methods of diagnosis are not practical for regular mass screening at short time intervals. Thermography could be a solution to this issue because it is a non-invasive and low-cost method that can be used routinely as a self-screening method. The research significance of this work lies in the implementation and integration of multiple different AI techniques for achieving diagnosis based on breast thermograms from several data sources. The data sources contain 306 images. The concept of transfer learning with several pre-trained models is implemented. Bayesian Networks (BNs) are also used to have interpretability of the diagnosis. A novel feature extraction from images (related to temperature) has been implemented and feeds the BNs. Finally, all methods and the classification results of pre-trained models are compared. It is found that the best result amongst the transfer learning concept is achieved with MobileNet, which delivered 93.8% accuracy. Furthermore, the BN achieves an accuracy of 90.20%, and finally, the expert model that combines CNNs and BNs gives an accuracy of 90.85%, even with a limited amount of data available. The integration of CNN and BN aims to overcome the hardship of interpretability. These approaches demonstrate high performance with added interpretability compared to previous works. In conclusion, the deep neural network provides promising results in breast cancer detection. It could be an ideal candidate for Breast Self-Exam (BSE), the goal recommended by WHO for mass screening.

Conversational search is the dominant intent of Question Answering, which is achieved through different NLP techniques, Deep Learning models. The advent of Transformer models has been a breakthrough in Natural Language Processing applications, which has attained a benchmark on state of NLP tasks such as question answering. Here we propose a semantic Malayalam Question Answering system that automatically answers the queries related to health issues. The Biomedical Question-Answering, especially in the Malayalam language, is a tedious and challenging task. The proposed model uses a neural network-based Bidirectional Encoder Representation from Transformers (BERT), to implement the question answering system. In this study, we investigate how to train and fine-tune a BERT model for Question-Answering. The system has been tested with our own annotated Malayalam SQUAD form health dataset. In comparing the result with our previous works - Word embedding and RNN based model, identified we find that our BERT model is more accurate than the previous models and achieves an F1 score of 86%.

This paper presents a study of the negative effect of Machine Translation (MT) on the precision of Cross---Lingual Question Answering (CL---QA). For this research, a English---Spanish Question Answering (QA) system is used. Also, the sets of 200 official questions from CLEF 2004 and 2006 are used. The CL experimental evaluation using MT reveals that the precision of the system drops around 30% with regard to the monolingual Spanish task. Our main contribution consists on a taxonomy of the identified errors caused by using MT and how the errors can be overcome by using our proposals. An experimental evaluation proves that our approach performs better than MT tools, at the same time contributing to this CL---QA system being ranked first at English---Spanish QA CLEF 2006.

Invited Papers.- Turning the Web into a Database: Extracting Data and Structure.- Analysis and Improvement of Minimally Supervised Machine Learning for Relation Extraction.- Information Retrieval.- Using Evidences Based on Natural Language to Drive the Process of Fusing Multimodal Sources.- An Experimental Comparison of Explicit Semantic Analysis Implementations for Cross-Language Retrieval.- Term Extraction.- CITOM: Incremental Construction of Topic Maps.- Schema-Matching with Data Dictionaries.- Requirements Analysis: Concept Extraction and Translation of Textual Specifications to Executable Models.- Information Extraction.- Score-Based Approach for Anaphora Resolution in Drug-Drug Interactions Documents.- Relation Extraction for Monitoring Economic Networks.- Classification of Text.- Real-Word Typo Detection.- The Impact of Semantic and Morphosyntactic Ambiguity on Automatic Humour Recognition.- Semantic Approaches to Fine and Coarse-Grained Feature-Based Opinion Mining.- Classification of Documents.- Analyzing Document Collections via Context-Aware Term Extraction.- AIR: A Semi-Automatic System for Archiving Institutional Repositories.- Protein-Protein Interactions Classification from Text via Local Learning with Class Priors.- Interfaces to Knowledge Bases.- Natural Language Interfaces: What Is the Problem? - A Data-Driven Quantitative Analysis.- Semantic Mapping between Natural Language Questions and SQL Queries via Syntactic Pairing.- Search Interface to a Mayan Glyph Database Based on Visual Characteristics.- Using Semantic Models of Natural Language.- Temporal Expression Identification Based on Semantic Roles.- Evaluation of Arabic Machine Translation System Based on the Universal Networking Language.- Quality Assessment of Knowledge Sources.- Quality of Subsumption Hierarchies in Ontologies.- Evaluating Semantic Relations by Exploring Ontologies on the Semantic Web.- On the Assessment of Text Corpora.- Poster Papers.- Improving Semantic Web Applications with Navigational Semantics.- Hybrid Algorithm for Word-Level Alignment of Parallel Texts.- A Hybrid Approach for Data Mart Schema Design from NL-OLAP Requirements.- Spreadsheet Information Retrieval through Natural Language.- Characterizing Weblog Corpora.- Improving Full Text Search with Text Mining Tools.- Cognos: A Natural Interaction Knowledge Management Toolkit.- Doctoral Symposium Papers.- Specialized Entailment Engines: Approaching Linguistic Aspects of Textual Entailment.- Natural Language Processing for Spreadsheet Information Retrieval.- Use Speech for User's Group Identification.

Number of people having expertise in a certain domain is less than people who need information in that domain. In this situation, an automatic question answering (QA) system is necessary. Observing available manual QA sites on internet, the real world question that people usually ask have different expected answer type (EAT) compared to a common automatic QA. Addressing a case study of a religion domain which makes it a closed domain QA, we proposed the EAT into 6 types: LAW, DEFINITION, COMPARISON, METHOD, TIME and PERSON. Different with common QA approach, we built the QA system using case based approach which consists of two main components: Question Analyzer and Case Retriever. Related with the case based reasoning (CBR) framework, these two main components act as the Retrieve and Reuse process while the Revise and Retain process is handle by Case Retainer component. The QA system was built using available Indonesian Natural Language Processing (NLP) tools and FreeCBR as the CBR library. The experiments were done to calculate the accuracy and testing the system with unknown case. By using 77 cases collected from internet with assumption that all answers are available, the experiments achieved 97% accuracy. And by using 10 test cases for the unknown case, the similarity score calculated by the system showed that the test questions have no answer in the available case base.

Prediction and classification of Alzheimer Disease categories using Integrated Deep Transfer Learning Approach

Manufacturing text often exists as unlabeled data; the entity is fine-grained and the extraction is difficult. The above problems mean that the manufacturing industry knowledge utilization rate is low. This paper proposes a novel Chinese fine-grained NER (named entity recognition) method based on symmetry lightweight deep multinetwork collaboration (ALBERT-AttBiLSTM-CRF) and model transfer considering active learning (MTAL) to research fine-grained named entity recognition of a few labeled Chinese textual data types. The method is divided into two stages. In the first stage, the ALBERT-AttBiLSTM-CRF was applied for verification in the CLUENER2020 dataset (Public dataset) to get a pretrained model; the experiments show that the model obtains an F1 score of 0.8962, which is better than the best baseline algorithm, an improvement of 9.2%. In the second stage, the pretrained model was transferred into the Manufacturing-NER dataset (our dataset), and we used the active learning strategy to optimize the model effect. The final F1 result of Manufacturing-NER was 0.8931 after the model transfer (it was higher than 0.8576 before the model transfer); so, this method represents an improvement of 3.55%. Our method effectively transfers the existing knowledge from public source data to scientific target data, solving the problem of named entity recognition with scarce labeled domain data, and proves its effectiveness.

Wind power prediction using deep neural network based meta regression and transfer learning

Previous works on evaluating the performance of Question Answering (QA) systems are focused on the evaluation of the precision. In this paper, we developed a mathematic procedure in order to explore new evaluation measures in QA systems considering the answer time. Also, we carried out an exercise for the evaluation of QA systems within a time constraint in the CLEF-2006 campaign, using the proposed measures. The main conclusion is that the evaluation of QA systems in realtime can be a new scenario for the evaluation of QA systems.

The concept of transfer learning has received a great deal of concern and interest throughout the last decade. Selecting an ideal representational framework for instances of various domains to minimize the divergence among source and target domains is a fundamental research challenge in representative transfer learning. The domain adaptation approach is designed to learn more robust or higher-level features, required in transfer learning. This paper presents a novel transfer learning framework that employs a marginal probability-based domain adaptation methodology followed by a deep autoencoder. The proposed frame adapts the source and target domain by plummeting distribution deviation between the features of both domains. Further, we adopt the deep neural network process to transfer learning and suggest a supervised learning algorithm based on encoding and decoding layer architecture. Moreover, we have proposed two different variants of the transfer learning techniques for classification, which are termed as (i) Domain adapted transfer learning with deep autoencoder-1 (D-TLDA-1) using the linear regression and (ii) Domain adapted transfer learning with deep autoencoder-2 (D-TLDA-2) using softmax regression. Simulations have been conducted with two popular real-world datasets: ImageNet datasets for image classification problem and 20_Newsgroups datasets for text classification problem. Experimental findings have established and the resulting improvements in accuracy measure of classification shows the supremacy of the proposed D-TLDA framework over prominent state-of-the-art machine learning and transfer learning approaches.