Contrastive knowledge integrated graph neural networks for Chinese medical text classification

Entity Alignment (EA) aims to identify equivalent entities across different Knowledge Graphs (KGs), enabling knowledge fusion and integration. In recent years, Graph Neural Networks (GNNs) have emerged as a powerful paradigm for EA by leveraging structural information in KGs. However, most existing studies emphasize novel message passing mechanisms while overlooking other crucial GNN design components. This paper presents a comprehensive and systematic evaluation of GNN-based EA methods, focusing on three key aspects: message passing strategies, the number of GNN layers, and the construction of final entity representations. We benchmark a diverse set of GNN models originally developed for tasks such as node classification and knowledge graph completion, and we assess their adaptability to the EA task. Additionally, we explore the effectiveness of skip connection techniques, activation functions, and relational information integration. Our experiments, conducted on standard EA benchmarks including DBP15K and SRPRS, reveal several counterintuitive findings: (1) message passing is indispensable for EA; (2) many node classification GNNs are highly competitive for EA; (3) one or two GNN layers generally achieve optimal performance; and (4) activation functions have minimal impact, while skip connections significantly enhance results. This study provides a principled framework and empirical foundation for designing more effective GNN-based EA models. Code and datasets are publicly available at <uri xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://github.com/kg-bnu/GNN-EA">https://github.com/kg-bnu/GNN-EA</uri>.

Knowledge graph fine-grained network with attribute transfer for recommendation

Text FCG: Fusing Contextual Information via Graph Learning for text classification

Recent studies have shown that the problems of cold processing and data sparsity in recommender systems can be well alleviated with the help of knowledge graph information. Meanwhile, since the information in recommender systems can be represented by graph data structure, graph neural network has great advantages in recommendatory system. Therefore, graph neural network can better mine the deep relationship between objects in the recommender systems which combines with information from knowledge graph. This paper reviews the application of graph neural network in knowledge graph aided recommendation systems. Specifically, this paper firstly sorts out the related knowledge of knowledge graph, recommendation system and graph neural network, including the cold processing and data sparsity, the structure of knowledge graph, and the basic modules of graph neural network. This paper also summarizes the methods such as KGAT and CKAN used in knowledge graph aided recommendation systems in recent years, and gives some discussion on the development of this field.

Modeling user preferences from users' historical sequences is one of the core problems of sequential recommendation. Previous studies only considered transition patterns between user-items, ignoring transition patterns between item features and item-item interactions. Recently, there has been interest in integrating knowledge graphs as auxiliary information into sequential recommendation. Most of the existing methods deal with the heterogeneous information in the knowledge graph in a coarse-grained manner. We believe that fine-grained processing of information in knowledge graphs can help recommendation systems understand changes in user preferences, i.e. dividing hetero-geneous information into item-to-item relationships and item-to-attribute relationships. In this paper, we propose a dual-level self-attention network for sequential recommendation. Specifically, we divide the knowledge graph heterogeneous information about items into relation level and attribute level, representing item-item relationship and item-attribute relationship, respectively. Afterwards, the self-attention network is used to learn user preferences at dual-level, respectively. Then, the outputs of the above dual levels are integrated for next item recommendation. Based on extensive experiments on three real-world data sets, our model achieves significant improvements compared to state-of-the-art baseline methods.

Multitype view of knowledge contrastive learning for recommendation

Existing knowledge graph-based recommendation algorithms either use Knowledge Graphs (KGs) as auxiliary information or use KG prediction tasks as regular terms to constrain recommendation with multi-task learning. However, the first method, which introduces multi-hop neighbors to enhance item representations, also weakens the relationship information within individual triples to some extent. The second method ignores neighboring information and thus fails to capture the long-range connectivity between items, which leads to a lack of utilization of the structural information in the KG. In response to the limitations of existing recommendation algorithms that do not fully leverage the relational and structural information within KGs, we propose a novel method named Knowledge-Enhanced Multi-Task Parallelized Recommendation Algorithm Incorporating Attention-Embedded Propagation (KMPR-AEP). It employs a parallel approach to simultaneously incorporate KG as auxiliary information and a regular term. We take into account the multi-hop neighbor information of users and items as well as enhance the relations among individual triples, which considers both the structural and relational information of KG. Extensive experiments conducted on three real-world datasets for CTR prediction and Top-K recommendation scenarios demonstrate the superiority of our proposed method over the state-of-the-art. On average, our KMPR-AEP shows improvements of 2.44% in AUC, 3.49% in ACC, and 2.64% in F1.

PurposeIn the last few years, the size of Linked Open Data (LOD) describing artworks, in general or domain-specific Knowledge Graphs (KGs), is gradually increasing. This provides (art-)historians and Cultural Heritage professionals with a wealth of information to explore. Specifically, structured data about iconographical and iconological (icon) aspects, i.e. information about the subjects, concepts and meanings of artworks, are extremely valuable for the state-of-the-art of computational tools, e.g. content recognition through computer vision. Nevertheless, a data quality evaluation for art domains, fundamental for data reuse, is still missing. The purpose of this study is filling this gap with an overview of art-historical data quality in current KGs with a focus on the icon aspects.Design/methodology/approachThis study’s analyses are based on established KG evaluation methodologies, adapted to the domain by addressing requirements from art historians’ theories. The authors first select several KGs according to Semantic Web principles. Then, the authors evaluate (1) their structures’ suitability to describe icon information through quantitative and qualitative assessment and (2) their content, qualitatively assessed in terms of correctness and completeness.FindingsThis study’s results reveal several issues on the current expression of icon information in KGs. The content evaluation shows that these domain-specific statements are generally correct but often not complete. The incompleteness is confirmed by the structure evaluation, which highlights the unsuitability of the KG schemas to describe icon information with the required granularity.Originality/valueThe main contribution of this work is an overview of the actual landscape of the icon information expressed in LOD. Therefore, it is valuable to cultural institutions by providing them a first domain-specific data quality evaluation. Since this study’s results suggest that the selected domain information is underrepresented in Semantic Web datasets, the authors highlight the need for the creation and fostering of such information to provide a more thorough art-historical dimension to LOD.

Designing intelligent systems to classify text in the medical domain is a challenging task. There is a shortage of openly available medical datasets (due to HIPPA-related strict regulations on protected health information for patients). In this paper, we explore the application of Open Source Medical LLMs (such as Meditron LLM), generic Large Language Models (such as LLAMA2), and Small Language Models (such as Phi2) on medical text classification (medical abstract dataset). We show that PEFT approaches such as LoRA can perform very well in classifying medical text, which involves interpreting patient conditions and symptoms and determining what medical problems the patients have. These approaches (based on Large and Small Language Models) have outperformed the current state of the results on medical abstracts corpus. In addition to medical LLMs, the open-source generic LLMs can be adapted to solving classification tasks on medical text and perform nearly as well as the specialized medical LLMs. SLMs can be a serious contender for solving domain-specific classification tasks (e.g., medical literature). This shows that carefully selecting the training data and fine-tuning positively impacts classification accuracy, precision, and recall. Generic Language Models such as LLAMA2 (LLM) and Phi2 (SLM) weren’t specifically trained with medical text. Medical LLMs such as Meditron outperform LLAMA2 and Phi2 in precision and accuracy. This is quite evident as Meditron was originally trained on medical text. The (micro averaged) F1 score for the fine-tuned Meditron model is 0.64. This is superior compared to fined-tuned LLAMA2 of 0.58 and Phi2 of 0.62. We see that Phi2 can outperform LLAMA2 with fewer number of parameters. The approaches used in this work can be extended to other medical text classification problems in the medical domain.

Knowledge Graph Embeddings (KGE) aim to map entities and relations to low dimensional spaces and have become the de-facto standard for knowledge graph completion. Most existing KGE methods suffer from the sparsity challenge, where it is harder to predict entities that appear less frequently in knowledge graphs. In this work, we propose a novel framework KRACL1 to alleviate the widespread sparsity in KGs with graph context and contrastive learning. Firstly, we propose the Knowledge Relational Attention Network (KRAT) to leverage the graph context by simultaneously projecting neighboring triples to different latent spaces and jointly aggregating messages with the attention mechanism. KRAT is capable of capturing the subtle semantic information and importance of different context triples as well as leveraging multi-hop information in knowledge graphs. Secondly, we propose the knowledge contrastive loss by combining the contrastive loss with cross entropy loss, which introduces more negative samples and thus enriches the feedback to sparse entities. Our experiments demonstrate that KRACL achieves superior results across various standard knowledge graph benchmarks, especially on WN18RR and NELL-995 which have large numbers of low in-degree entities. Extensive experiments also bear out KRACL’s effectiveness in handling sparse knowledge graphs and robustness against noisy triples.

Knowledge graphs display various entities and their relationships in the real world based on knowledge representation and can analyze and predict the intrinsic relationship between knowledge embeddings through data mining and information processing, which are widely used in search engines, web analytics and smart recommendation areas. As existing knowledge graph information is continuously created and grown, it is a curial task to determine whether the information in knowledge graph is correct and to complete the missing information. In response to this challenge, the researchers proposed a number of graph convolutional network (GCN)-based models to characterize knowledge graphs. The state-of-the-art model is R-GCN, which can effectively extract features. This paper deeply studies the algorithm ideas and results of the R-GCN model, explores whether the model can be further optimized in entity classification and link prediction, and finally, improves the original model and proposes a relational aggregation graph convolutional network. Specifically, this paper finds that a subset of the set of entities may be directly connected to a central entity. All the entities in this subset possess partially identical attributes. At the same time, the relationships between these entities and the central entity may be similar. These similar attributes and relationships can be abstractly aggregated into virtual entities and virtual relationships, respectively, to better extract the topological relationship features. This paper uses the FB15k dataset to evaluate the performance of the proposed model on knowledge graph completion tasks. The experimental results show that the proposed RA-GCN model achieves a certain level improvement compared with the original model and that it can extract knowledge graph topological relationship characteristics more effectively.

The automatic identification of the topology of power networks is important for the data-driven and situation-aware operation of power grids. Traditional methods of topology identification lack a data-tolerant mechanism, and the accuracy of their performance in terms of identification is thus affected by the quality of data. Topology identification is related to the link prediction problem. The graph neural network can be used to predict the state of unlabeled nodes (lines) through training on features of labeled nodes (lines) with fault tolerance. Inspired by the characteristics of the graph neural network, we applied it to topology identification in this study. We propose a method to identify the topology of a power network based on a knowledge graph and the graph neural network. Traditional knowledge graphs can quickly mine possible connections between entities and generate graph structure data, but in the case of errors or informational conflicts in the data, they cannot accurately determine whether the relationships between the entities really exist. The graph neural network can use data mining to determine whether a connection obtained between entities is based on their eigenvalues, and has a fault tolerance mechanism to adapt to errors and informational conflicts in the graph data, but needs the graph data as database. The combination of the knowledge graph and the graph neural network can compensate for the deficiency of the single knowledge graph method. We tested the proposed method by using the IEEE 118-bus system and a provincial network system. The results showed that our approach is feasible and highly fault tolerant. It can accurately identify network topology even in the presence of conflicting and missing measurement-related information.

A knowledge graph (KG) consists of numerous triples, in which each triple, i.e., (head entity, relation, tail entity), denotes a real-world assertion. Many large-scale KGs have been developed, e.g., general-purpose KGs Freebase and YAGO. Also, lots of domain-specific KGs are emerging, e.g., COVID-19 KGs, biomedical KGs, and agricultural KGs. By embedding KGs into low-dimensional vectors, i.e., representations of entities and relations, we could integrate KGs into machine learning models and enhance the performance of many prediction tasks, including search, recommendations, and question answering. During the construction, refinement, embedding, and application of KGs, a variety of KG learning algorithms have been developed to handle challenges in various real-world scenarios. Moreover, graph neural networks have also brought new opportunities to KG learning. This workshop aims to engage with active researchers from KG communities, recommendation communities, natural language processing communities, and other communities, and deliver state-of-the-art research insights into the core challenges in KG learning.

Clinical decision support system for hypertension medication based on knowledge graph

Purpose Path information in knowledge graphs can provide explicit explanations for recommendation decisions, thus becoming a focus in explainable recommendation research. However, limited studies about explainable recommendation problems result in low model transparency and poor persuasiveness, affecting user experience. Therefore, the goal is to provide accurate and interpretable recommendations for recommendations. Design/methodology/approach This study proposes a recommendation reasoning method based on knowledge graph and reinforcement learning. To alleviate the noise problem in the state space, a multi-head attention mechanism is used to learn state expressions. A dual critic network is used to optimize long-term and short-term rewards simultaneously, achieving path reasoning in the knowledge graph to provide short-term and long-term value explanations for recommendation decisions. Findings We conduct extensive experiments on the real-world benchmark dataset and the domain dataset to validate the effectiveness of our method on the recommendation and explanation tasks, which proves the method's ability to generate high-quality and interpretable course recommendations. Originality/value Developing explainable recommendation methods based on the combination of knowledge graph and reinforcement learning is crucial to overcome the current limitations. A recommendation system that integrates knowledge reasoning, autonomous learning and interpretability may meet the needs of the modern education field for explainable recommendation systems.