Mining Ontological Knowledge from Domain-Specific Text Documents

Model tuning or prompt Tuning? a study of large language models for clinical concept and relation extraction

At present, the focus of ontology learning is on the extraction of concepts and relations. The relation extraction is divided into hierarchical relation extraction and non-hierarchical relation extraction, and hierarchical relation extraction is the basis of non-hierarchical relation extraction. In this paper, the method of mixed hierarchical clustering is used to extract and classify domain concepts from textual corpora in different fields of Uyghur language. Due to the limitations of the text content selected in this paper and the difficulty of merging nodes by hierarchical clustering, although the results are not ideal, they basically conform to the ontology hierarchy form. The experimental results show that the method is feasible, and the factors such as increasing the vector dimension and the text content, can further improve the accuracy of the domain concept and relation.

BackgroundDiabetic retinopathy (DR) is a leading cause of blindness in American adults. If detected, DR can be treated to prevent further damage causing blindness. There is an increasing interest in developing artificial intelligence (AI) technologies to help detect DR using electronic health records. The lesion-related information documented in fundus image reports is a valuable resource that could help diagnoses of DR in clinical decision support systems. However, most studies for AI-based DR diagnoses are mainly based on medical images; there is limited studies to explore the lesion-related information captured in the free text image reports.MethodsIn this study, we examined two state-of-the-art transformer-based natural language processing (NLP) models, including BERT and RoBERTa, compared them with a recurrent neural network implemented using Long short-term memory (LSTM) to extract DR-related concepts from clinical narratives. We identified four different categories of DR-related clinical concepts including lesions, eye parts, laterality, and severity, developed annotation guidelines, annotated a DR-corpus of 536 image reports, and developed transformer-based NLP models for clinical concept extraction and relation extraction. We also examined the relation extraction under two settings including ‘gold-standard’ setting—where gold-standard concepts were used–and end-to-end setting.ResultsFor concept extraction, the BERT model pretrained with the MIMIC III dataset achieve the best performance (0.9503 and 0.9645 for strict/lenient evaluation). For relation extraction, BERT model pretrained using general English text achieved the best strict/lenient F1-score of 0.9316. The end-to-end system, BERT_general_e2e, achieved the best strict/lenient F1-score of 0.8578 and 0.8881, respectively. Another end-to-end system based on the RoBERTa architecture, RoBERTa_general_e2e, also achieved the same performance as BERT_general_e2e in strict scores.ConclusionsThis study demonstrated the efficiency of transformer-based NLP models for clinical concept extraction and relation extraction. Our results show that it’s necessary to pretrain transformer models using clinical text to optimize the performance for clinical concept extraction. Whereas, for relation extraction, transformers pretrained using general English text perform better.

Scientific publications are often associated with a set of keywords to describe their content. Automating the process of keyword extraction and assignment could be useful in indexing electronic documents and building digital libraries. In this paper we propose a new approach to biomedical Concept Extraction (CE) using semantic features of concept graphs. We represent full-text documents by graphs and map biomedical terms to predefined ontology concepts. We adopt concept relation weights to improve the ranking process of potential key concepts. We perform both objective and human-based subjective evaluations. The results show that using relation weights significantly improves the performance of CE. The results also highlight the subjectivity of the CE procedure as well as of its evaluation.

Ontology plays a very important role in supporting knowledge-based applications. In cloud computing, ontology learning technology is facing new challenges in dealing with heterogeneous data sources from different domains and researchers, which may contain various particular concepts and relations. Traditional ontology learning frameworks usually focus only on the extraction of concepts and taxonomic relations from the multi-structured corpus. However, former researches rarely studied the interactions during ontology learning process among different researchers. Lack of interactions among people who build ontology in different domains may cause inconsistent ontology. Besides, lack of incentive during the ontology building process will also result in low efficiency. To address these challenges, this paper specifies a novel solution to perform ontology learning. The solution includes a service-oriented ontology interaction framework, a service-oriented ontology learning strategy. It shows that it advances ontology learning to a higher level of performance and portability with a number of experiments in demo system.

A large number of medical research papers and clinical notes on disease diagnostic, treatment and prevention are increasing every day. This biomedical text provides a rich source of knowledge for biomedical research. However, this medical information is scattered in vast medical informatics literature in unstructured form. It is requisite to retrieve imperative information from these publications and discover new knowledge. A lot of research is done in biomedical text mining using different methods and techniques. Centre of i2b2 organized different challenges on natural language processing for medical text. In i2b2 2010, challenge tasks were focused on concept extraction, assertion classification and relation extraction, and in 2012, the task was temporal information extraction. In previous work, various machine learning techniques are found to be one of the effective techniques to extract clinical information from different types of medical data like discharge summary, physical notes. This paper presents the review of earlier work on different machine learning techniques and methods for medical research. The effectiveness of these techniques has been measured by precision, recall and F-score. This review will be useful for biomedical researchers to identify best techniques for the further research in clinical information extraction.

Extracting cybersecurity entities and the relationships between them from online textual resources such as articles, bulletins, and blogs and converting these resources into more structured and formal representations has important applications in cybersecurity research and is valuable for professional practitioners. Previous works to accomplish this task were mainly based on utilizing feature-based models. Feature-based models are time-consuming and need labor-intensive feature engineering to describe the properties of entities, domain knowledge, entity context, and linguistic characteristics. Therefore, to alleviate the need for feature engineering, we propose the usage of neural network models, specifically the long short-term memory (LSTM) models to accomplish the tasks of Named Entity Recognition (NER) and Relation Extraction (RE). We evaluated the proposed models on two tasks. The first task is performing NER and evaluating the results against the state-of-the-art Conditional Random Fields (CRFs) method. The second task is performing RE using three LSTM models and comparing their results to assess which model is more suitable for the domain of cybersecurity. The proposed models achieved competitive performance with less feature-engineering work. We demonstrate that exploiting neural network models in cybersecurity text mining is effective and practical.

Nowadays, ontologies have been exploited in many current applications due to the abilities in representing knowledge and inferring new knowledge. However, the manual construction of ontologies is tedious and time-consuming. Therefore, the automated ontology construction from text has been investigated. The extraction of taxonomic relations between concepts is a crucial step in constructing domain ontologies. To obtain taxonomic relations from a text corpus, especially when the data is deficient, the approach of using the web as a source of collective knowledge (a.k.a web-based approach) is usually applied. The important challenge of this approach is how to collect relevant knowledge from a large amount of web pages. To overcome this issue, we propose a framework that combines Word Sense Disambiguation (WSD) and web approach to extract taxonomic relations from a domain-text corpus. This framework consists of two main stages: concept extraction and taxonomic-relation extraction. Concepts acquired from the concept-extraction stage are disambiguated through WSD module and passed to stage of extraction taxonomic relations afterward. To evaluate the efficiency of the proposed framework, we conduct experiments on datasets about two domains of tourism and sport. The obtained results show that the proposed method is efficient in corpora which are insufficient or have no training data. Besides, the proposed method outperforms the state of the art method in corpora having high WSD results.

The goal of medical concept extraction is to identify phrases that refer to medical concepts of interest such as problems, treatments and tests from medical documents. In this study, three types of medical concept extraction models are developed and then compared them. The first concept extraction task is mainly based upon semantic features obtained from a domain-knowledge based method using MetaMap, and the other two are machine-learning methods with using sequential classifier Conditional Random Fields (CRF) for both with and without MetaMap outputs as features. Among the three concept extraction models, the combined approach of CRF with MetaMap features obtained the best results.

Ontology learning from unstructured text has become a critical task for knowledge-driven applications in Big Data and Artificial Intelligence. While significant advances have been made in the automatic extraction of concepts and relations using neural and Transformer-based models, the generation of formal Description Logic axioms required for constructing logically consistent and computationally tractable ontologies remains largely underexplored. This paper puts forward a novel pipeline for automated axiom generation through schema mapping. Our paper introduces three key innovations: a deterministic mapping framework that guarantees logical consistency (unlike stochastic Large Language Models); guaranteed formal consistency verified by OWL reasoners (unaddressed by prior statistical methods); and a transparent, scalable bridge from neural extractions to symbolic logic, eliminating manual post-processing. Technically, the pipeline builds upon the outputs of a Transformer-based fusion model for joint concept and relation extraction. We then map lexical relational phrases to formal ontological properties through a lemmatization-based schema alignment step. Entity typing and hierarchical induction are then employed to infer class structures, as well as domain and range constraints. Using RDFLib and structured data processing, we transform the extracted triples into both assertional (ABox) and terminological (TBox) axioms expressed in Description Logic. Experimental evaluation on benchmark datasets (Conll04 and NYT) demonstrates the efficacy of the approach, with expert validation showing high acceptance rates (>95%) and reasoners confirming zero inconsistencies. The pipeline thus establishes a reliable, scalable foundation for automated ontology learning, advancing the field from extraction to formally verifiable knowledge base construction.

Joint entity and relation extraction based on a hybrid neural network

Relation extraction is a research hot-spot in the field of natural language processing, and aims at structured knowledge acquirement. However, existing methods still grapple with the issue of entity overlapping, where they treat relation types as inconsequential labels, overlooking the fact that relation type has a great influence on entity type hindering the performance of these models from further improving. Furthermore, current models are inadequate in handling the fine-grained aspect of entity positioning, which leads to ambiguity in entity boundary localization and uncertainty in relation inference, directly. In response to this challenge, a relation extraction model is proposed, which is guided by relational semantic cues and focused on entity boundary localization. The model uses an attention mechanism to align relation semantics with sentence information, so as to obtain the most relevant semantic expression to the target relation instance. It then incorporates an entity locator to harness additional positional features, thereby, enhancing the capability of the model to pinpoint entity start and end tags. Consequently, this approach effectively alleviates the problem of entity overlapping. Extensive experiments are conducted on the widely used datasets NYT and WebNLG. The experimental results show that the proposed model outperforms the baseline ones in F1 scores of the two datasets, and the improvement margin is up to 5.50% and 2.80%, respectively.

Relation Extraction (RE) is one of the fundamental tasks in Information Extraction. The goal of this task is to find the semantic relations between entity mentions in text. It has been shown in many previous work that the structure of the sentences (i.e., dependency trees) can provide important information/features for the RE models. However, the common limitation of the previous work on RE is the reliance on some external parsers to obtain the syntactic trees for the sentence structures. On the one hand, it is not guaranteed that the independent external parsers can offer the optimal sentence structures for RE and the customized structures for RE might help to further improve the performance. On the other hand, the quality of the external parsers might suffer when applied to different domains, thus also affecting the performance of the RE models on such domains. In order to overcome this issue, we introduce a novel method for RE that simultaneously induces the structures and predicts the relations for the input sentences, thus avoiding the external parsers and potentially leading to better sentence structures for RE. Our general strategy to learn the RE-specific structures is to apply two different methods to infer the structures for the input sentences (i.e., two views). We then introduce several mechanisms to encourage the structure and semantic consistencies between these two views so the effective structure and semantic representations for RE can emerge. We perform extensive experiments on the ACE 2005 and SemEval 2010 datasets to demonstrate the advantages of the proposed method, leading to the state-of-the-art performance on such datasets.

Extracting information from the medical imaging reports with domain ontology has attracted much attention in medical natural language processing field. Based on the unstructured characteristics of medical image report text, the existing image report domain ontology is constructed by ontology learning method, including ontology learning method based on language and ontology learning method based on machine learning. However, these existing methods ignore the anatomical knowledge embedded in medical imaging reports, which is very useful for extracting entity relationships from reports. In order to solve the above problems, this paper proposes a domain ontology construction method for medical image reporting based on anatomy knowledge, which combines the prior knowledge of pathology and anatomy to obtain the basic framework of the domain ontology as the knowledge driver. In particular, our proposed approach consists of two tasks. The first task is to convert each text report into a semantic tree through an anatomic knowledge based semantic subtree generation algorithm. Semantic subtree generation algorithm mainly consists of three parts: framework positioning, relation extraction and adding relation. Firstly, the text is located based on anatomical knowledge, and the relational extraction mainly adopts the method of dependency syntactic analysis to extract the three semantic relations contained in the text. Finally, add the relationship to the corresponding branch. The second task is to obtain the domain ontology by merging semantic subtree. This step is mainly to obtain the domain ontology by merging the nodes of the XML structure semantic tree. The experimental results show that this method can be used for relational extraction and domain ontology construction, laying a good foundation for the follow-up research.

The World Wide Web includes semantic relations of numerous types that exist among different entities. Extracting the relations that exist between two entities is an important step in various Web-related tasks such as information retrieval (IR), information extraction, and social network extraction. A supervised relation extraction system that is trained to extract a particular relation type (source relation) might not accurately extract a new type of a relation (target relation) for which it has not been trained. However, it is costly to create training data manually for every new relation type that one might want to extract. We propose a method to adapt an existing relation extraction system to extract new relation types with minimum supervision. Our proposed method comprises two stages: learning a lower dimensional projection between different relations, and learning a relational classifier for the target relation type with instance sampling. First, to represent a semantic relation that exists between two entities, we extract lexical and syntactic patterns from contexts in which those two entities co-occur. Then, we construct a bipartite graph between relation-specific (RS) and relation-independent (RI) patterns. Spectral clustering is performed on the bipartite graph to compute a lower dimensional projection. Second, we train a classifier for the target relation type using a small number of labeled instances. To account for the lack of target relation training instances, we present a one-sided under sampling method. We evaluate the proposed method using a data set that contains 2,000 instances for 20 different relation types. Our experimental results show that the proposed method achieves a statistically significant macroaverage F-score of 62.77. Moreover, the proposed method outperforms numerous baselines and a previously proposed weakly supervised relation extraction method.