A Hierarchical Clustering Based Relation Extraction Method for Domain Ontology

Model tuning or prompt Tuning? a study of large language models for clinical concept and relation extraction

Application of hierarchical clustering, singularity mapping, and Kohonen neural network to identify Ag-Au-Pb-Zn polymetallic mineralization associated geochemical anomaly in Pangxidong district

Traditional text mining systems employ shallow parsing techniques and focus on concept extraction and taxonomic relation extraction. This paper presents a novel system called CRCTOL for mining rich semantic knowledge in the form of ontology from domain-specific text documents. By using a full text parsing technique and incorporating both statistical and lexico-syntactic methods, the knowledge extracted by our system is more concise and contains a richer semantics compared with alternative systems. We conduct a case study wherein CRCTOL extracts ontological knowledge, specifically key concepts and semantic relations, from a terrorism domain text collection. Quantitative evaluation, by comparing with a state-of-the-art ontology learning system known as text-to-onto, has shown that CRCTOL produces much better precision and recall for both concept and relation extraction, especially from sentences with complex structures.

Nowadays, ontologies have been exploited in many current applications due to the abilities in representing knowledge and inferring new knowledge. However, the manual construction of ontologies is tedious and time-consuming. Therefore, the automated ontology construction from text has been investigated. The extraction of taxonomic relations between concepts is a crucial step in constructing domain ontologies. To obtain taxonomic relations from a text corpus, especially when the data is deficient, the approach of using the web as a source of collective knowledge (a.k.a web-based approach) is usually applied. The important challenge of this approach is how to collect relevant knowledge from a large amount of web pages. To overcome this issue, we propose a framework that combines Word Sense Disambiguation (WSD) and web approach to extract taxonomic relations from a domain-text corpus. This framework consists of two main stages: concept extraction and taxonomic-relation extraction. Concepts acquired from the concept-extraction stage are disambiguated through WSD module and passed to stage of extraction taxonomic relations afterward. To evaluate the efficiency of the proposed framework, we conduct experiments on datasets about two domains of tourism and sport. The obtained results show that the proposed method is efficient in corpora which are insufficient or have no training data. Besides, the proposed method outperforms the state of the art method in corpora having high WSD results.

Community welfare is one of the important points for a region and is also the essence of national development. The welfare of the people in Indonesia is fairly unequal, especially in East Java. To be able to map an area to the welfare of its people in East Java, one way that can be used is to use clustering. The hierarchical clustering method is one of the clustering methods for grouping data. In hierarchical clustering, single linkage, complete linkage, and average linkage methods are suitable methods for grouping data, which will compare the best method to use. The results of the calculation show that the average linkage method with three clusters is the best calculation with a silhouette index value of 0.6054, with the 1st cluster there are 23 regions, namely the city/district with the highest community welfare, the 2nd cluster there are 11 regions, namely cities/districts with moderate social welfare, and in the third cluster there are 4 regions, namely cities/districts with the lowest community welfare.

BackgroundDiabetic retinopathy (DR) is a leading cause of blindness in American adults. If detected, DR can be treated to prevent further damage causing blindness. There is an increasing interest in developing artificial intelligence (AI) technologies to help detect DR using electronic health records. The lesion-related information documented in fundus image reports is a valuable resource that could help diagnoses of DR in clinical decision support systems. However, most studies for AI-based DR diagnoses are mainly based on medical images; there is limited studies to explore the lesion-related information captured in the free text image reports.MethodsIn this study, we examined two state-of-the-art transformer-based natural language processing (NLP) models, including BERT and RoBERTa, compared them with a recurrent neural network implemented using Long short-term memory (LSTM) to extract DR-related concepts from clinical narratives. We identified four different categories of DR-related clinical concepts including lesions, eye parts, laterality, and severity, developed annotation guidelines, annotated a DR-corpus of 536 image reports, and developed transformer-based NLP models for clinical concept extraction and relation extraction. We also examined the relation extraction under two settings including ‘gold-standard’ setting—where gold-standard concepts were used–and end-to-end setting.ResultsFor concept extraction, the BERT model pretrained with the MIMIC III dataset achieve the best performance (0.9503 and 0.9645 for strict/lenient evaluation). For relation extraction, BERT model pretrained using general English text achieved the best strict/lenient F1-score of 0.9316. The end-to-end system, BERT_general_e2e, achieved the best strict/lenient F1-score of 0.8578 and 0.8881, respectively. Another end-to-end system based on the RoBERTa architecture, RoBERTa_general_e2e, also achieved the same performance as BERT_general_e2e in strict scores.ConclusionsThis study demonstrated the efficiency of transformer-based NLP models for clinical concept extraction and relation extraction. Our results show that it’s necessary to pretrain transformer models using clinical text to optimize the performance for clinical concept extraction. Whereas, for relation extraction, transformers pretrained using general English text perform better.

Hierarchical clustering, a traditional clustering method, has been getting attention again. Among several reasons, a credit goes to a recent paper by Dasgupta in 2016 that proposed a cost function that quantitatively evaluates hierarchical clustering trees. An important question is how to combine this recent advance with existing successful clustering methods. In this paper, we propose a hierarchical clustering method to minimize the cost function of clustering tree by incorporating existing clustering techniques. First, we developed an ensemble tree-search method that finds an integrated tree with reduced cost by integrating multiple existing hierarchical clustering methods. Second, to operate on large and arbitrary shape data, we designed an efficient hierarchical clustering framework, called integrating divisive and ensemble-agglomerate (IDEA) by combining it with advanced clustering techniques such as nearest neighbor graph construction, divisive-agglomerate hybridization, and dynamic cut tree. The IDEA clustering method showed better performance in minimizing Dasgupta's cost and improving accuracy (adjusted rand index) over existing cost-minimization-based, and density-based hierarchical clustering methods in experiments using arbitrary shape datasets and complex biology-domain datasets.

Supplementary Material for the manuscript "Drivers of carbon dioxide emissions: An empirical investigation using hierarchical and non- hierarchical clustering methods"

Ontology plays a very important role in supporting knowledge-based applications. In cloud computing, ontology learning technology is facing new challenges in dealing with heterogeneous data sources from different domains and researchers, which may contain various particular concepts and relations. Traditional ontology learning frameworks usually focus only on the extraction of concepts and taxonomic relations from the multi-structured corpus. However, former researches rarely studied the interactions during ontology learning process among different researchers. Lack of interactions among people who build ontology in different domains may cause inconsistent ontology. Besides, lack of incentive during the ontology building process will also result in low efficiency. To address these challenges, this paper specifies a novel solution to perform ontology learning. The solution includes a service-oriented ontology interaction framework, a service-oriented ontology learning strategy. It shows that it advances ontology learning to a higher level of performance and portability with a number of experiments in demo system.

Clustering is a group of unsupervised statistical techniques commonly used in many disciplines. Considering their applications to fish abundance data, many technical details need to be considered to ensure reasonable interpretation. However, the reliability and stability of the clustering methods have rarely been studied in the contexts of fisheries. This study presents an intensive evaluation of three common clustering methods, including hierarchical clustering (HC), K-means (KM), and expectation-maximization (EM) methods, based on fish community surveys in the coastal waters of Shandong, China. We evaluated the performances of these three methods considering different numbers of clusters, data size, and data transformation approaches, focusing on the consistency validation using the index of average proportion of non-overlap (APN). The results indicate that the three methods tend to be inconsistent in the optimal number of clusters. EM showed relatively better performances to avoid unbalanced classification, whereas HC and KM provided more stable clustering results. Data transformation including scaling, square-root, and log-transformation had substantial influences on the clustering results, especially for KM. Moreover, transformation also influenced clustering stability, wherein scaling tended to provide a stable solution at the same number of clusters. The APN values indicated improved stability with increasing data size, and the effect leveled off over 70 samples in general and most quickly in EM. We conclude that the best clustering method can be chosen depending on the aim of the study and the number of clusters. In general, KM is relatively robust in our tests. We also provide recommendations for future application of clustering analyses. This study is helpful to ensure the credibility of the application and interpretation of clustering methods.

The purpose of data clustering algorithm is to form clusters (groups) of data points such that there is high intra-cluster and low inter-cluster similarity. There are different types of clustering methods such as hierarchical, partitioning, grid and density based. Hierarchical clustering is a method of cluster analysis which seeks to build a hierarchy of clusters. A hierarchical clustering method can be thought of as a set of ordinary (flat) clustering methods organized in a tree structure. These methods construct the clusters by recursively partitioning the objects in either a top-down or bottom-up fashion. In this paper we present a new hierarchical clustering algorithm using Euclidean distance. To validate this method we have performed some experiments with low dimensional artificial datasets and high dimensional fMRI dataset. Finally the result of our method is compared to some of existing clustering methods.

Extracting cybersecurity entities and the relationships between them from online textual resources such as articles, bulletins, and blogs and converting these resources into more structured and formal representations has important applications in cybersecurity research and is valuable for professional practitioners. Previous works to accomplish this task were mainly based on utilizing feature-based models. Feature-based models are time-consuming and need labor-intensive feature engineering to describe the properties of entities, domain knowledge, entity context, and linguistic characteristics. Therefore, to alleviate the need for feature engineering, we propose the usage of neural network models, specifically the long short-term memory (LSTM) models to accomplish the tasks of Named Entity Recognition (NER) and Relation Extraction (RE). We evaluated the proposed models on two tasks. The first task is performing NER and evaluating the results against the state-of-the-art Conditional Random Fields (CRFs) method. The second task is performing RE using three LSTM models and comparing their results to assess which model is more suitable for the domain of cybersecurity. The proposed models achieved competitive performance with less feature-engineering work. We demonstrate that exploiting neural network models in cybersecurity text mining is effective and practical.

The aim of this research is to design a Total Productive Maintenance (TPM) management model using a hybrid method of artificial neural networks and hierarchical clustering in power distribution companies of northwestern Iran. This study is conducted in power distribution companies of northwestern Iran, which were selected as pilot companies. Determining the optimal maintenance strategy and selecting the best management model for maintenance is of great importance. The findings of this study will be provided to Tavanir and the Ministry of Energy for further implementation in other subsidiary companies. In terms of location, the quantitative data pertains to operational data of the power distribution companies in northwestern Iran. The statistical population includes experts and personnel from the maintenance, repair, and warehousing departments of these companies. The temporal data pertains to operational data from the inventory, accounting, and process systems of power distribution companies in northwestern Iran, spanning from 2017 to 2022. The results of this research indicate that initiating the Total Productive Maintenance process requires strong managerial leadership. Subsequently, processes should be improved and undergo initial feedback evaluations. By considering the strength of human resources and enhancing employee skills, the quality of work processes will be analyzed. As these factors evolve, the system will undergo precise organization and planning. Comprehensive preventive maintenance will ensure workplace safety and health are prioritized. Another aspect that management must address is the advancement of technology and the expansion of automation systems, especially in the implementation of equipment and inventory management subsystems and resource and contract management, which are key priorities of the model. Finally, management must focus on adopting preventive maintenance, self-controlled maintenance, and re-evaluating current practices. Employees should be engaged in achieving these three goals.

Distant supervision is an efficient approach for various tasks, such as relation extraction. Most of the recent literature on distantly supervised relation extraction generates labeled data by heuristically aligning knowledge bases with text corpora and then trains supervised relation classification models based on statistical learning. However, extracting long tail relations from the automatically labeled data is still a challenging problem even in big data. Inspired by explanation-based learning (EBL), this paper proposes an EBL-based approach to tackle this problem. The proposed approach can learn relation extraction rules effectively using unlabeled data. Experiments on the New York Times corpus demonstrate that our approach outperforms the baseline approach especially on long tail data.

Incidents can occur anytime and anywhere whether intentional or unintentional, especially in the industrial area. To prevent the incident, an analysis is needed to find out the patterns that formed based on the information from incident reports. The goals of company are to be able to identify and take action to deal quickly if some incident happens. Text Mining is done by applying the bigram technique to form the incident patterns. The analytical method used is K-Means Clustering and Hierarchical Clustering. In addition, a feature selection is also used by applying the Genetic Algorithm method to obtain optimal features. The results obtained state that the feature selection process is very influential on the formation of incident clusters. When compared, the K-Means Clustering and Hierarchical Clustering methods have different effects on each warehouse sector. The best results in the Sector of Life Style and Sector of Technology are formed by using the K-Means Clustering method while in the Sector of Consumer, Retail and SPL the best cluster results are obtained based on the Hierarchical Clustering method.