CASBERT: BERT-based retrieval for compositely annotated biosimulation model entities

With the tremendous growth in the number of scientific papers being published, searching for references while writing a scientific paper is a time-consuming process. A technique that could add a reference citation at the appropriate place in a sentence will be beneficial. In this perspective, context-aware citation recommendation has been researched upon for around two decades. Many researchers have utilized the text data called the context sentence, which surrounds the citation tag, and the metadata of the target paper to find the appropriate cited research. However, the lack of well-organized benchmarking datasets and no model that can attain high performance has made the research difficult. In this paper, we propose a deep learning based model and well-organized dataset for context-aware paper citation recommendation. Our model comprises a document encoder and a context encoder, which uses Graph Convolutional Networks (GCN) layer and Bidirectional Encoder Representations from Transformers (BERT), which is a pre-trained model of textual data. By modifying the related PeerRead dataset, we propose a new dataset called FullTextPeerRead containing context sentences to cited references and paper metadata. To the best of our knowledge, This dataset is the first well-organized dataset for context-aware paper recommendation. The results indicate that the proposed model with the proposed datasets can attain state-of-the-art performance and achieve a more than 28% improvement in mean average precision (MAP) and recall@k.

<p>These are the conference proceedings of COMBINE 2019, the 10th Computational Modeling in Biology Network (COMBINE) meeting that took place in Heidelberg (Germany) from July 15th to July 19th, 2019.</p> <p>The Computational Modeling in Biology Network (COMBINE) is an initiative to coordinate the development of the various community standards and formats for computational models in the life sciences. It was created in 2010 to enable the sharing of resources, tools, and other infrastructure, and to coordinate standardization efforts for modeling in biology. COMBINE brings standard communities together around activities that are mutually beneficial. These activities include making specification documents available from a common location, providing a central point of contact, and organizing regular face-to-face meetings. To this end, the COMBINE network organizes an annual conference style meeting (the COMBINE Forum) and annual hackathon style events called HARMONY (Hackathon on Resources for Modeling in Biology), as well as tutorials and training events.</p> <p>At COMBINE 2019 a combination of keynote lectures, invited talks and interactive breakout discussions, as well as contributed talks, posters and lightning talks, selected from submitted abstracts, provided the basis for the meeting, offering diverse formats to exchange information, to discuss and work on interoperability problems and to demonstrate support for standards implemented in modelling tools, platforms and databases. One special focus of COMBINE 2019 was on the standardization need in systems medicine, which has been recognized as a necessity for computer-assisted personalized medicine. To direct attention to this topic, the European standardization framework for data integration and data-driven in silico models (EUSTANDS4PM) organized a workshop as part of COMBINE 2019. Also, reproducibility in modelling was a main topic, as reflected by several sessions and workshops. Besides these focus themes, also sessions, workshops and breakout discussions around single standards, their further development and their interoperability helped to advance the standardization, standing in the tradition of COMBINE.</p>

The relationship between food and disease holds significant importance in traditional medicine and healthcare practices. However, when it comes to disseminating dietary and health-related information, the news media often falls short in terms of authority and accuracy. To address this issue, this paper introduces a method for extracting relationships among food, herbals, diseases, and symptoms leveraging BERT (Bidirectional Encoder Representations from Transformers) framework. The method utilizes Transfer Learning to fine-tune the BERT pretraining model on an existing dataset to enable the model to recognize entities such as food, herbals, diseases, symptoms from unstructured text found in various literature sources. Following entity recognition, an entity embedding masking process is employed to construct a tailored training dataset specifically designed for entity relationship extraction. By combining entity recognition and relationship extraction, this method effectively uncovers the connections between food, herbals, diseases, and symptoms. The experimental results: Named Entity Recognition precision rate of 93.51%, a recall of 92.22%, and F1 of 92.86%. Relation Extraction accuracy of 91.99%, F1 of 93.88%. This method addresses the scarcity of annotated data in this domain, offering a method for automatic extraction of entity relationships, even without the need for expert annotations.

Providing reliable health information to a community can help raise awareness of the dangers of diseases, their causes, methods of prevention, and treatment. Indonesians are facing various health problems partly due to the lack of health information; hence, the community needs media that can effectively provide reliable health information, namely a question answering (QA) system. The frequently asked questions are non-factoid questions. The development of answer selection based on the classical approach requires distinctive engineering features, linguistic tools, or external resources. It can be solved using deep learning approach such as Convolutional Neural Networks (CNN). However, this model cannot capture the sequence of words in both questions and answers. Therefore, this study aims to implement a long short-term memory (LSTM) model to effectively exploit long-range sequential context information for an answer selection task. In addition, this study analyses various hyper-parameters of Word2Vec and LSTM, such as the dimension, context window, dropout, hidden unit, learning rate, and margin; the corresponding values that yield the best mean reciprocal rank (MRR) and mean average precision (MAP) are found to be 300, 15, 0.25, 100, 0.01, and 0.1, respectively. The best model yields MAP and MRR values of 82.05% and 91.58%, respectively. These results experienced an increase in MAP and MRR of 18.68% and 46.11%, respectively, compared to CNN as the baseline model.

Rapid increase in scholarly publications on the web has posed a new challenge to the researchers in finding highly relevant and important research articles associated with a particular area of interest. Even a highly relevant paper is sometimes missed especially for novice researchers due to lack of knowledge and experience in finding and accessing the most suitable articles. Scholarly recommender system is a very appropriate tool for this purpose that can enable researchers to locate relevant publications easily and quickly. However, the main downside of the existing approaches is that their effectiveness is dependent on priori user profiles and thus, they cannot recommend papers to the new users. Furthermore, the system uses both public and non-public metadata and therefore, the system is unable to find similarities between papers efficiently due to copyright restrictions. Considering the above challenges, in this research work, a novel hybrid approach is proposed that separately combines a Content Based Filtering (CBF) recommender module and a Collaborative Filtering (CF) recommender module. Unlike previous CBF and CF approaches, public contextual metadata and paper-citation relationship information are effectively incorporated into these two approaches separately to enhance the recommendation accuracy. In order to verify the effectiveness of the proposed approach, publicly available datasets were employed. Experimental results demonstrate that the proposed approach outperforms the baseline approaches in terms of standard metrics (precision, recall, F1-measure, mean average precision, and mean reciprocal rank), indicating that the proposed approach is more efficient in recommending scholarly publications.

This chapter introduces information retrieval (IR)-based fault localization techniques. One of the challenges of using bug tracking systems is that they often contain too many bug reports for developers to handle. Due to the textual nature of bug reports, IR techniques are often employed to solve the problem, and many IR-based fault localization techniques have been proposed in literature. A common fault localization process consists of four steps, that is, corpus creation, indexing, query construction, and retrieval and ranking. Top- k prediction accuracy, mean reciprocal rank, and mean average precision are widely used to evaluate the performance of IR-based fault localization techniques. The chapter aims to categorize studies on IR-based fault localization into three groups: text of current bug report only, text and history, and text and stack/execution traces. Existing IR-based fault localization techniques can be categorized based on data used in the fault localization process or IR techniques used.

To address the problem of sparse relationships between entities in the current knowledge graph and the increase in complexity and computational effort in knowledge inference, this paper proposes a knowledge inference method based on a multi-headed attention mechanism - the MAtt-BiGRU model. For a given triad, the path information is encoded by a two-way gated recurrent neural network, and the confidence of the model is calculated by aggregating it with the candidate paths with the help of the multi-headed attention mechanism, while the inference ability of the model is enhanced by dynamic negative sample training. Numerical experiments are conducted on the MAtt-BiGRU model using large knowledge graph datasets NELL-995 and FB15k-237. The experimental results show that the proposed MAtt-BiGRU model has better results in Mean Average Precision (MAP) and Mean Reciprocal Rank(MRR) performance evaluation metrics when compared with existing commonly used path ranking algorithms such as Path Ranking Algorithm(PRA) and Path-RNN, and the proposed model can more accurately reason about the relations between entities.

The field of expert finding has seen a large number of approaches proposed both in universities and in industries, using a variety of new techniques in relevant data fields. This study tends to identify information retrieval method of latent semantic analysis and temporal graph for document retrieval. In this study, citation occurrence and author occurrence are independent variables and scales of expert author finding are dependent variables. The method used to evaluate judgment of document and author relevance in the test set formation phase is more similar to survey methods. Library method is used to study theoretical foundations and judge literature. This study has three populations: a) test set documents; b) people who make queries and judge relevance of retrieved documents; c) people who judge relevance of the retrieved experts. To measure judgments of document relevance, a method similar to peer tests is used. Among the retrieved results, repeated results are placed to determine accuracy and reliability of the judge. The degree of correlation obtained in this method is very high (0.98), indicating the reliability of the results. Regarding the results of the current study on application of latent semantic indexing (LSA) information retrieval model, which was ultimately used to retrieve expert authors, the performance of LSA-based retrieval model outperformed the baseline model. This was evident from the obtained metrics, including precision at the top 5 results (p@5) with a value of 0.895, mean average precision (MAP) of 0.839, and mean reciprocal rank (MRR) of 0.909. The improved retrieval performance can be attributed to the superior performance of the dimension reduction method compared to keyword matching.

Answer selection aims to identify the most relevant answers to a given question from a set of candidates. It is the fundamental component of intelligent question answering system. To improve performance, it gradually becomes an effective strategy to integrate external structured knowledge bases (KBs) into the answer selection model. Due to expensive cost of construction and maintenance of such KBs, these models are suffering from domain barriers and information incompleteness. In this paper, we propose a two-stage extraction–comprehension answer selection model, which can extract ad-hoc knowledge from unstructured web texts to enhance the performance of answer selection. For the extraction, two types of snippets are extracted from unstructured web pages and utilized as the source of ad-hoc knowledge. For the comprehension, a selective attention mechanism is employed to extract and integrate ad-hoc knowledge from multiple text snippets obtained in the first stage, which can enrich the representation of question–answer pairs and more accurately identify the correct answers. By incorporating ad-hoc knowledge extracted from both types of snippets, the proposed model achieves state-of-the-art results on two public available benchmark datasets. In particular, on WikiQA, in terms of the two evaluation metrics (mean average precision and mean reciprocal rank), it achieves 9.9[Formula: see text] and 8.4[Formula: see text] higher than the previous non-pretraining-based models, and 3.4[Formula: see text] and 3.2[Formula: see text] higher than the pretraining-based models.

Accurate principal discharge diagnoses are fundamental for effective morbidity surveillance, yet healthcare systems worldwide are plagued by imprecise and non-specific coding, hindering public health analysis and resource allocation. This article introduces a novel computational model designed to address this challenge by automatically predicting specific discharge diagnoses from a combination of structured and unstructured clinical data. Our approach utilizes a dual-pipeline architecture to process heterogeneous information streams. The structured data pipeline systematically extracts diagnostic indicators from laboratory results, pharmacy records, and surgical procedure codes (CUPS). Concurrently, the unstructured data pipeline employs a sophisticated multi-stage natural language processing (NLP) framework to interpret free-text radiology reports and clinical notes. This NLP pipeline first uses a fine-tuned bidirectional encoder representations from transformers (BERT) model to accurately detect and filter out negated or uncertain statements. Subsequently, a Sentence-Transformer model performs semantic similarity analysis to match affirmative clinical findings against a curated, expert-validated knowledge base of diagnostic sentences. For novel or complex cases not covered by the knowledge base, the system leverages the generative capabilities of GPT-3.5 Turbo as a fallback mechanism. The model is implemented as a scalable, service-oriented application using a MySQL database and a FastAPI framework, ensuring seamless integration with existing hospital electronic health record (EHR) systems. Performance was rigorously evaluated using a custom 0-5 scoring metric, developed in collaboration with a thoracic surgery specialist and validated by a team of three hospital epidemiologists, to assess clinical accuracy. Results demonstrate high performance in identifying both primary and secondary diagnoses, with notable success in common critical conditions like pulmonary embolism. Furthermore, the model shows significant potential for scalability, as performance on rare conditions such as epidural hematoma improved substantially with the addition of only a few expert-defined diagnostic patterns. This open-source computational tool offers a robust and economically viable solution for healthcare institutions to enhance the precision of their diagnostic coding, thereby improving the quality of morbidity data without requiring disruptive changes to established clinical workflows.

Software bugs are a noteworthy concern for developers and maintainers. When a failure is detected late, it costs more to be fixed. To repair the bug that caused the software failure, the location of the bug must first be known. The process of finding the defective source code elements that led to the failure of the software is called bug localization. Effective approaches for automatically locating bugs using bug reports are highly desirable, as they would reduce bug-fixing time, consequently lowering software maintenance costs. With the increasing size and complexity of software projects, manual bug localization methods have become complex, challenging, and time-consuming tasks, which motivates research on automated bug localization techniques. This paper introduces a novel bug localization model, which works on two levels. The first level localizes the buggy classes using an information retrieval approach and it has two additional sub-phases, namely the class-level feature scoring phase and the class-level final score and ranking phase, which ranks the top buggy classes. The second level localizes the buggy methods inside these classes using an information retrieval approach and it has two sub-phases, which are the method-level feature scoring phase and the method-level final score and ranking phase, which ranks the top buggy methods inside the localized classes. A model is evaluated using an AspectJ dataset, and it can correctly localize and rank more than 350 classes and more than 136 methods. The evaluation results show that the proposed model outperforms several state-of-the-art approaches in terms of the mean reciprocal rank (MRR) metrics and the mean average precision (MAP) in class-level bug localization.

This study presents the results of experimental evaluations of an ontology-based frequently asked question retrieval system in the domain of HIV and AIDS. The main purpose of the system is to provide answers to questions on HIV/AIDS using ontology. To evaluate the effectiveness of the frequently asked question retrieval system, we conducted two experiments. The first experiment focused on the evaluation of the quality of the ontology we developed using the OQuaRE evaluation framework which is based on software quality metrics and metrics designed for ontology quality evaluation. The second experiment focused on evaluating the effectiveness of the ontology in retrieving relevant answers. For this we used an open-source information retrieval platform, Terrier, with retrieval models BM25 and PL2. For the measurement of performance, we used the measures mean average precision, mean reciprocal rank, and precision at 5. The results suggest that frequently asked question retrieval with ontology is more effective than frequently asked question retrieval without ontology in the domain of HIV/AIDS.

Circular RNA (circRNA) is a closed-loop structural non-coding RNA molecule which plays a significant role during the gene regulation processes. There are many previous studies shown that circRNAs can be regarded as the sponges of miRNAs. Thus, circRNA is also a key point for disease diagnosing, treating and inferring. However, traditional experimental approaches to verify the associations between the circRNA and disease are time-consuming and money-consuming. There are few computational models to predict potential circRNA-disease associations, which become our motivation to propose a new computational model. In this study, we propose a machine learning based computational model named Gradient Boosting Decision Tree with multiple biological data to predict circRNA-disease associations (GBDTCDA). The known circRNA-disease associations' data are downloaded from cricR2Disease database (http://bioinfo.snnu.edu.cn/CircR2Disease/). The feature vector of each circRNA-disease association pair is composed of four parts, which are the statistics information of different biological networks, the graph theory information of different biological networks, circRNA-disease associations' network information and circRNA nucleotide sequence information, respectively. Therefore, we use those feature vectors to train the gradient boosting decision tree regression model. Then, the leave one out cross validation (LOOCV) is adopted to evaluate the performance of our computational model. As for predicting some common diseases related circRNAs, our method GBDTCDA also obtains the better results. The Area under the ROC Curve (AUC) values of Basal cell carcinoma, Non-small cell lung cancer and cervical cancer are 95.8%, 88.3% and 93.5%, respectively. For further illustrating the performance of GBDTCDA, a case study of breast cancer is also supplemented in this study. Thus, our proposed method GBDTCDA is a powerful tool to predict potential circRNA-disease associations based on experimental results and analyses.

Purpose: The peer review process is essential for maintaining the quality of scientific publications. However, identifying reviewers who possess the necessary expertise can be challenging. In Open Journal Systems (OJS), which is commonly utilized by journals, the most effective method of inviting reviewers is when they are already registered in the system. This study seeks to improve the efficiency and accuracy of the reviewer selection process to ensure high-quality peer reviews.Methods: We introduced a process innovation to analyze users within OJS and obtain recommendations for potential reviewers possessing the relevant expertise for the manuscript under review. This study collected user data from OJS as potential reviewers and utilized information from the Scopus search application programming interface (API). We extracted authors’ data from the Scopus API to obtain their Scopus IDs, which were then used to scrape publication data of potential reviewers. The system matched the previous works of reviewers with the title and abstract of the manuscript using term frequency-inverse document frequency and cosine similarity algorithms.Results: The system was evaluated by comparing its recommendations with the assessments made by the editorial team. This evaluation yielded precision, mean average precision, and mean reciprocal rank values of 0.47, 0.77, and 0.87, respectively.Conclusion: The results clearly demonstrate the system’s ability to provide relevant reviewer recommendations. This system offers significant benefits by assisting editors in identifying suitable reviewer candidates from the existing user database in OJS, particularly for the evaluation of manuscripts.

Children’s acquisition of language is an amazing feat. Children master the syntax, the sentence structure of their language, through exposure and interaction with caregivers and others but, notably, with no formal tuition. How children come to be in command of the syntax of their language has been a topic of vigorous debate since Chomsky argued against Skinner’s claim that language is ‘verbal behavior.’ Chomsky argued that knowledge of language cannot be learned through experience alone but is guided by a genetic component. This language component, known as ‘Universal Grammar,’ is composed of abstract linguistic knowledge and a computational system that is special to language. The computational mechanisms of Universal Grammar give even young children the capacity to form hierarchical syntactic representations for the sentences they hear and produce. The abstract knowledge of language guides children’s hypotheses as they interact with the language input in their environment, ensuring they progress toward the adult grammar. An alternative school of thought denies the existence of a dedicated language component, arguing that knowledge of syntax is learned entirely through interactions with speakers of the language. Such ‘usage-based’ linguistic theories assume that language learning employs the same learning mechanisms that are used by other cognitive systems. Usage-based accounts of language development view children’s earliest productions as rote-learned phrases that lack internal structure. Knowledge of linguistic structure emerges gradually and in a piecemeal fashion, with frequency playing a large role in the order of emergence for different syntactic structures.