Enhancing named entity recognition in low-resource scenarios using diffusion variational autoencoder and knowledge graphs

Integrating the Knowledge Graphs (KGs) into recommendation systems enhances personalization and accuracy. However, the long-tail distribution of knowledge graphs often leads to data sparsity, which limits the effectiveness in practical applications. To address this challenge, this study proposes a knowledge-aware recommendation algorithm framework that incorporates multi-level contrastive learning. This framework enhances the Collaborative Knowledge Graph (CKG) through a random edge dropout method, which constructs feature representations at three levels: user-user interactions, item-item interactions and user-item interactions. A dynamic attention mechanism is employed in the Graph Attention Networks (GAT) for modeling the KG. Combined with the nonlinear transformation and Momentum Contrast (Moco) strategy for contrastive learning, it can effectively extract high-quality feature information. Additionally, multi-level contrastive learning, as an auxiliary self-supervised task, is jointly trained with the primary supervised task, which further enhances recommendation performance. Experimental results on the MovieLens and Amazon-books datasets demonstrate that this framework effectively improves the performance of knowledge graph-based recommendations, addresses the issue of data sparsity, and outperforms other baseline models across multiple evaluation metrics.

Although there are a small number of work to conduct patent research by building knowledge graph, but without constructing patent knowledge graph using patent documents and combining latest natural language processing methods to mine hidden rich semantic relationships in existing patents and predict new possible patents. In this paper, we propose a new patent vacancy prediction approach named PatentMiner to mine rich semantic knowledge and predict new potential patents based on knowledge graph (KG) and graph attention mechanism. Firstly, patent knowledge graph over time (e.g. year) is constructed by carrying out named entity recognition and relation extrac-tion from patent documents. Secondly, Common Neighbor Method (CNM), Graph Attention Networks (GAT) and Context-enhanced Graph Attention Networks (CGAT) are proposed to perform link prediction in the constructed knowledge graph to dig out the potential triples. Finally, patents are defined on the knowledge graph by means of co-occurrence relationship, that is, each patent is represented as a fully connected subgraph containing all its entities and co-occurrence relationships of the patent in the knowledge graph; Furthermore, we propose a new patent prediction task which predicts a fully connected subgraph with newly added prediction links as a new pa-tent. The experimental results demonstrate that our proposed patent predic-tion approach can correctly predict new patents and Context-enhanced Graph Attention Networks is much better than the baseline. Meanwhile, our proposed patent vacancy prediction task still has significant room to im-prove.

Event extraction as machine reading comprehension with question-context bridging

Biomedical knowledge graphs (KGs), which can help with the understanding of complex biological systems and pathologies, have begun to play a critical role in medical practice and research. However, challenges remain in their embedding and use due to their complex nature and the specific demands of their construction. Existing studies often suffer from problems such as sparse and noisy datasets, insufficient modeling methods and non-uniform evaluation metrics. In this work, we established a comprehensive KG system for the biomedical field in an attempt to bridge the gap. Here, we introduced PharmKG, a multi-relational, attributed biomedical KG, composed of more than 500000 individual interconnections between genes, drugs and diseases, with 29 relation types over a vocabulary of ~8000 disambiguated entities. Each entity in PharmKG is attached with heterogeneous, domain-specific information obtained from multi-omics data, i.e. gene expression, chemical structure and disease word embedding, while preserving the semantic and biomedical features. For baselines, we offered nine state-of-the-art KG embedding (KGE) approaches and a new biological, intuitive, graph neural network-based KGE method that uses a combination of both global network structure and heterogeneous domain features. Based on the proposed benchmark, we conducted extensive experiments to assess these KGE models using multiple evaluation metrics. Finally, we discussed our observations across various downstream biological tasks and provide insights and guidelines for how to use a KG in biomedicine. We hope that the unprecedented quality and diversity of PharmKG will lead to advances in biomedical KG construction, embedding and application.

This paper describes our approach for the Chinese medical Named Entity Recognition (NER) and event extraction tasks organized by the China Conference on Knowledge Graph and Semantic Computing (CCKS) 2021. For the NER task, we need to identify the entity boundaries and category labels of six types of entities from Chinese electronic medical records (EMR). And for the Event Extraction task, we need to recognizes a type of tumor event from Chinese EMR, which contains three tumor-related attributes. This medical entity and event extraction task has two main challenges: 1) How to build a unified modeling framework for entity and event extraction. 2) How to improve the generalization ability of medical entity extraction. For these two challenges, we use a sequence labeling framework based on entity extraction to unify the above two tasks. Based on the pre-trained model, we propose a combined strategy of unsupervised text mode enhancement and label mode enhancement. In the end, it ranked second without any post-processing.KeywordsNamed Entity RecognitionEvent extractionElectronic medical records

Medical named entity recognition via lattice-enhanced transfer learning with random attention

Predicting stock trends in financial markets is of significant importance to investors and portfolio managers. In addition to a stock’s historical price information, the correlation between that stock and others can also provide valuable information for forecasting future returns. Existing methods often fall short of straightforward and effective capture of the intricate interdependencies between stocks. In this research, we introduce the concept of a Laplacian correlation graph (LOG), designed to explicitly model the correlations in stock price changes as the edges of a graph. After constructing the LOG, we will build a machine learning model, such as a graph attention network (GAT), and incorporate the LOG into the loss term. This innovative loss term is designed to empower the neural network to learn and leverage price correlations among different stocks in a straightforward but effective manner. The advantage of a Laplacian matrix is that matrix operation form is more suitable for current machine learning frameworks, thus achieving high computational efficiency and simpler model representation. Experimental results demonstrate improvements across multiple evaluation metrics using our LOG. Incorporating our LOG into five base machine learning models consistently enhances their predictive performance. Furthermore, backtesting results reveal superior returns and information ratios, underscoring the practical implications of our approach for real-world investment decisions. Our study addresses the limitations of existing methods that miss the correlation between stocks or fail to model correlation in a simple and effective way, and the proposed LOG emerges as a promising tool for stock returns prediction, offering enhanced predictive accuracy and improved investment outcomes.

The low-resource named entity recognition (NER) task faces the dual challenges of sparse annotation data and under-utilised syntactic structures. In this paper, we propose the Syntax-CLNP model, which solves the above problems through two core innovations: firstly, we design the Syntax-Aware Adapter, which dynamically injects dependent syntactic features by using differentiable syntactic projectors to realise semantic-syntactic collaborative characterisation; secondly, we propose the Contrastive Prototype Alignment (CPA) mechanism, which integrates contrastive learning and prototype network to strengthen the syntactic structure by gradient decoupling and enhance the syntactic structure through gradient decoupling. Prototype Alignment (CPA) mechanism is proposed, which integrates contrastive learning and prototype network, and strengthens intra-class feature compactness and inter-class separation through gradient decoupling to suppress noise interference. The experimental results show that on low resource datasets such as LingNER-40 (40 languages) and WikiANN, the F1 value of Syntax-CLNP is improved by 8.7% on average compared with the baseline model (p<0.01), the number of parameters increases by only 1% (111M), and the increase in inference latency is less than 4% (8.5ms), which achieves a balance between performance and efficiency. In the cross-language migration test, the model achieves 58.9% F1 in low-resource scenarios such as Tamil (<1k samples), verifying the generalisation capability. The study achieves the first end-to-end joint optimisation of syntactic tree and representation learning, providing an efficient solution for low-resource multilingual NER tasks.

MAGED: Multimodal attentive graph learning with gene expression dynamics on knowledge graphs for TCM target prediction.

Background Chinese patent medicines are increasingly used clinically, and the prescription drug monitoring program is an effective tool to promote drug safety and maintain health. Methods We constructed a prescription drug monitoring program for Chinese patent medicines based on knowledge graphs. First, we extracted the key information of Chinese patent medicines, diseases, and symptoms from the domain-specific corpus by the information extraction. Second, based on the extracted entities and relationships, a knowledge graph was constructed to form a rule base for the monitoring of data. Then, the named entity recognition model extracted the key information from the electronic medical record to be monitored and matched the knowledge graph to realize the monitoring of the Chinese patent medicines in the prescription. Results Named entity recognition based on the pretrained model achieved an F1 value of 83.3% on the Chinese patent medicines dataset. On the basis of entity recognition technology and knowledge graph, we implemented a prescription drug monitoring program for Chinese patent medicines. The accuracy rate of combined medication monitoring of three or more drugs of the program increased from 68% to 86.4%. The accuracy rate of drug control monitoring increased from 70% to 97%. The response time for conflicting prescriptions with two drugs was shortened from 1.3S to 0.8S. The response time for conflicting prescriptions with three or more drugs was shortened from 5.2S to 1.4S. Conclusions The program constructed in this study can respond quickly and improve the efficiency of monitoring prescriptions. It is of great significance to ensure the safety of patients' medication.

The accuracy of entity recognition is particularly important for knowledge graph construction. The traditional Named Entity recognition (NER) model mainly includes HMM, CRF, BiLSTM, BiLSTM-CRF, etc. It is difficult to solve the problem of word meaning confusion resulting from the wrong separation at the end of the entity when using the four models for the labeled nuclear technology knowledge data sets. In order to improve the entity recognition effect and address the problem of polysemy in nuclear technology knowledge data set, an improved nuclear technology entity recognition method based on the BERT-BiLSTM-CRF combination model was proposed by comparative experiments. According to the results, it can conclude that the application of the BERT model instead of the Word2Vec algorithm for word vector training is helpful to the model recognition, and the exclusive dictionary word segmentation and part of speech classification of nuclear technology texts contribute to improving the quality of labeled data. The experiment verified that the usage of BERT pre-training model can solve the problem of polysemy in NER to some extent. Meanwhile, it validated the precision in nuclear technology knowledge entity recognition for collected data, which could be used in pre-procedure in nuclear technology knowledge graph construction.

Knowledge graph, as a semantic network, provides a method to model knowledge from the perspective of entities and their relationships. Since a large quantity of material knowledge is widely distributed, using the structure of the knowledge graph is an effective approach to clarify the internal connections between related knowledge. Named entity recognition is a crucial step to construct a knowledge graph. However, most of the professional domain datasets are label-free and collected from multiple heterogeneous sources. To identify the entities from these datasets, a rule-based and deep learning hybrid method is proposed. The knowledge extracted by regular expression was used to semi-automatically label the corpus, and then trained by deep learning model, which achieved good results. The constructed knowledge graph regarding to the commonly used steels is of great significance to the integration and management of related knowledge, and it lays a foundation for future engineering application.KeywordsCommonly used steelKnowledge graphSemi-automatic labelingNamed entity recognition

A knowledge graph method for hazardous chemical management: Ontology design and entity identification

The introduction of knowledge graph as the auxiliary information of recommendation system provides a new research idea for personalized intelligent recommendation. However, most of the existing knowledge graph recommendation algorithms fail to effectively solve the problem of unrelated entities, leading to inaccurate prediction of potential preferences of users. To solve this problem, this paper proposes a KG-IGAT model combining knowledge graph and graph attention network, and adds an interest evolution module to graph attention network to capture user interest changes and generate top-N recommendations. Finally, experimental comparison between the proposed model and other algorithms using public data sets shows that KG-IGAT has better recommendation performance.

The intelligent development of continuous casting quality engineering is an essential step for the efficient production of high-quality billets. However, there are many quality defects that require strong expertise for handling. In order to reduce reliance on expert experience and improve the intelligent management level of billet quality knowledge, we focus on constructing a Domain-Specific Knowledge Graph (DSKG) for the quality engineering of continuous casting. To achieve joint extraction of billet quality defects entity and relation, we propose a Self-Attention Partition and Recombination Model (SAPRM). SAPRM divides domain-specific sentences into three parts: entity-related, relation-related, and shared features, which are specifically for Named Entity Recognition (NER) and Relation Extraction (RE) tasks. Furthermore, for issues of entity ambiguity and repetition in triples, we propose a semi-supervised incremental learning method for knowledge alignment, where we leverage adversarial training to enhance the performance of knowledge alignment. In the experiment, in the knowledge extraction part, the NER and RE precision of our model achieved 86.7% and 79.48%, respectively. RE precision improved by 20.83% compared to the baseline with sequence labeling method. Additionally, in the knowledge alignment part, the precision of our model reached 99.29%, representing a 1.42% improvement over baseline methods. Consequently, the proposed model with the partition mechanism can effectively extract domain knowledge, cand the semi-supervised method can take advantage of unlabeled triples. Our method can adapt the domain features and construct a high-quality knowledge graph for the quality engineering of continuous casting, providing an efficient solution for billet defect issues.