Research on Entity Relationship Joint Extraction Technology Based on Multitask Learning

The relationship between food and disease holds significant importance in traditional medicine and healthcare practices. However, when it comes to disseminating dietary and health-related information, the news media often falls short in terms of authority and accuracy. To address this issue, this paper introduces a method for extracting relationships among food, herbals, diseases, and symptoms leveraging BERT (Bidirectional Encoder Representations from Transformers) framework. The method utilizes Transfer Learning to fine-tune the BERT pretraining model on an existing dataset to enable the model to recognize entities such as food, herbals, diseases, symptoms from unstructured text found in various literature sources. Following entity recognition, an entity embedding masking process is employed to construct a tailored training dataset specifically designed for entity relationship extraction. By combining entity recognition and relationship extraction, this method effectively uncovers the connections between food, herbals, diseases, and symptoms. The experimental results: Named Entity Recognition precision rate of 93.51%, a recall of 92.22%, and F1 of 92.86%. Relation Extraction accuracy of 91.99%, F1 of 93.88%. This method addresses the scarcity of annotated data in this domain, offering a method for automatic extraction of entity relationships, even without the need for expert annotations.

The escalating frequency and severity of cyber-attacks have presented formidable challenges to the safeguarding of cyberspace. Named Entity Recognition (NER) technology is utilized for the rapid identification of threat entities and their relationships within cyber threat intelligence, enabling security researchers to be promptly informed of the occurrence of cyber threats, thereby enhancing the efficiency of security defense and analysis. However, current models for identifying network threat entities and extracting relationships suffer from limitations such as the inadequate representation of textual semantic information, insufficient granularity in threat entity recognition, and errors in relationship extraction propagation. To address these issues, this article proposes a novel model for Network Threat Entity Recognition and Relationship Extraction (CtiErRe). Additionally, it redefines seven network threat entities and two types of relationships between threat entities. Specifically, first, domain knowledge is collected to build a domain knowledge graph, which is then embedded using graph convolutional networks (GCN) to enhance the feature representation of threat intelligence text. Next, the features from domain knowledge graph embedding and those generated by the bidirectional encoder representations from transformers (BERT) model are fused using the Layernorm algorithm. Finally, the fused features are processed using the GlobalPointer algorithm to generate both the threat entity type matrix and the threat entity relation type matrix, thereby enabling the identification of threat entities and their relationships. To validate our proposed model, we conducted extensive experiments, and the results demonstrate its superiority over existing models. Our model performs remarkably in threat entity recognition tasks, with accuracy and F1 scores reaching 92.13% and 93.11%, respectively. In the relationship extraction task, our model achieves accuracy and F1 scores of 91.45% and 92.45%, respectively.

The Entity Relationship Extraction task involves accurately extracting target entities and uncovering their complex relational interdependencies within unstructured text, thereby establishing foundational structural knowledge for downstream applications including knowledge graph construction and semantic network population. While conventional approaches predominantly focus on sentence-level entity relationship extraction in general-domain texts. However, with the continuous expansion of practical needs, the task of entity relationship extraction no longer merely satisfies the extraction of relationships between entities from a single sentence. Instead, it requires accurately identifying the complex relationships between entities from multiple sentences.meanwhile, in the context of the Electric-power industry, in the task of entity relationship extraction in the electric professional field, due to the presence of a large number of professional terms and specific expressions in the texts of the power professional field, the relationships between entities are more complex and difficult to define. Using previous methods cannot accurately identify the entities in the power professional field and their relationships from the texts. To address these challenges, we propose a new framework for entity-relationship extraction in the electric domain based on large language models and modular fine-tuning, which we have named the R-E-M(Relationship-Entity Matching) framework. This framework comprises four components. First, it enhances the large language model's capability to recognize domainspecific entities in the power industry through targeted modular fine-tuning. Next, an entity-relationship extraction module identifies relevant entities and their corresponding relationships within the text. Subsequently, the entity-relationship matching module establishes predefined matching patterns between entities and relationships for each identified relation type. Finally, the prompt selection module dynamically chooses appropriate prompts based on different relationship types while specifying standardized output formats for the model. From the final experimental results, our proposed new framework has significantly improved the large language model's performance in the entity-relationship extraction task for power domain texts, elevating the Micro-F1 score from 0.185 to 0.575 -a substantial gain of 0.39, demonstrating remarkable enhancement.

The National Association of Stock Car Auto Racing (NASCAR) is ranked among the top ten most popular sports in the United States. NASCAR events are characterized by on-track racing punctuated by pit stops since cars must refuel, replace tires, and modify their setup throughout a race. A well-executed pit stop can allow drivers to gain multiple seconds on their opponents. Strategies around when to pit and what to perform during a pit stop are under constant evaluation. One currently unexplored area is publically available communication between each driver and their pit crew during the race. Due to the many hours of audio, manual analysis of even one driver’s communications is prohibitive. We propose a fully automated approach to analyze driver–pit crew communication. Our work was conducted in collaboration with NASCAR domain experts. Audio communication is converted to text and summarized using cluster-based Latent Dirichlet Analysis to provide an overview of a driver’s race performance. The transcript is then analyzed to extract important events related to pit stops and driving balance: understeer (pushing) or oversteer (over-rotating). Named entity recognition (NER) and relationship extraction provide context to each event. A combination of the race summary, events, and real-time race data provided by NASCAR are presented using Sankey visualizations. Statistical analysis and evaluation by our domain expert collaborators confirmed we can accurately identify important race events and driver interactions, presented in a novel way to provide useful, important, and efficient summaries and event highlights for race preparation and in-race decision-making.

Facing large-scale and rapidly growing material science literature data, text mining has become a research hotspot of material science. In recent years, natural language processing technology and machine learning methods have become the main technical means of text mining in materials science. The main task of text mining is to transform unstructured text data into structured material data by information extraction methods such as Named Entity Recognition and entity relationship extraction. This research proposes a general solution framework for material information extraction tasks, and introduces the main concepts and processes of text processing, text annotation, entity relationship extraction, etc., and discusses the current research progress and possible future research directions.

The traditional topic-based auditing model lacks the ability of multi-object and multi-abnormal correlation auditing, which makes it impossible to solve the multi-scenario and multi-factor correlation-based auditing problem. This paper designs an intelligent power marketing audit model based on a knowledge graph. First, an entity identification and relationship extraction method for power marketing business based on NLP (natural language processing) and sequence annotation technology is proposed, and the description content is imported into the knowledge graph database; then, semantic disambiguation and knowledge are carried out by using bidirectional encoder representation from transformers (BERT). Link to build a knowledge map of business audit rules: Finally, an experimental analysis is carried out by taking the copying and receiving business with a large business volume in the marketing audit work as an example, and it is verified that the proposed model can effectively improve the information analysis ability and the audit accuracy of the audit work.

Multitask learning has shown promising performance in learning multiple related tasks simultaneously, and variants of model architectures have been proposed, especially for supervised classification problems. One goal of multitask learning is to extract a good representation that sufficiently captures the relevant part of the input about the output for each learning task. To achieve this objective, in this paper we design a multitask learning architecture based on the observation that correlations exist between outputs of some related tasks (e.g. entity recognition and relation extraction tasks), and they reflect the relevant features that need to be extracted from the input. As outputs are unobserved, our proposed model exploits task predictions in lower layers of the neural model, also referred to as early predictions in this work. But we control the injection of early predictions to ensure that we extract good task-specific representations for classification. We refer to this model as a Progressive Multitask learning model with Explicit Interactions (PMEI). Extensive experiments on multiple benchmark datasets produce state-of-the-art results on the joint entity and relation extraction task.

The relationship between entity pairs is the key to understanding an entire sentence or paragraph in a natural language, so entity relationship extraction is particularly important in natural language processing tasks. When we understand the information of a sentence, we often need to combine the information into the context. The entity relationship of a sentence is expressed as a relationship, but combined with the next sentence, the entity represents another relationship,and It called negative feedback. In this paper, to solve these problems, we propose NFBATT.a entity relationship extraction model based negative feedback attention.which is based on improved attention and pre-training of BERT.The main is highlighted the negative feedback in attention of Distant Supervision Relation extraction,and in order to get a better vector representation get a better vector representation of semantics, the sentence vector we represent is a combination of word Embedding, entity Embedding and positional embedding of solid and non-solid entity words by deep learning, which we call sentence multi-information representation. We use the negative feedback attention to reduce the influence of negative feedback on sentence relationship discrimination. At the same time, we have also done a lot of experiments to prove it. Through extensive experiments on Sogou Person Knowledge Map dataset.we demonstrate NFBATT's effectiveness.and the accuracy rate is 82.0%, which is better than the best method currently available.

Few-shot learning under the <inline-formula><tex-math notation="LaTeX">$N$</tex-math></inline-formula> -way <inline-formula><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -shot setting (i.e., <inline-formula><tex-math notation="LaTeX">$K$</tex-math></inline-formula> annotated samples for each of <inline-formula><tex-math notation="LaTeX">$N$</tex-math></inline-formula> classes) has been widely studied in relation extraction (e.g., FewRel) and image classification (e.g., Mini-ImageNet). Named entity recognition (NER) is typically framed as a sequence labeling problem where the entity classes are inherently entangled together because the entity number and classes in a sentence are not known in advance, leaving the <inline-formula><tex-math notation="LaTeX">$N$</tex-math></inline-formula> -way <inline-formula><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -shot NER problem so far unexplored. In this paper, we first formally define a more suitable <inline-formula><tex-math notation="LaTeX">$N$</tex-math></inline-formula> -way <inline-formula><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -shot setting for NER. Then we propose <small>FewNER</small> , a novel meta-learning approach for few-shot NER. <small>FewNER</small> separates the entire network into a task-independent part and a task-specific part. During training in <small>FewNER</small> , the task-independent part is meta-learned across multiple tasks and the task-specific part is learned for each individual task in a low-dimensional space. At test time, <small>FewNER</small> keeps the task-independent part fixed and adapts to a new task via gradient descent by updating only the task-specific part, resulting in it being less prone to overfitting and more computationally efficient. Compared with pre-trained language models (e.g., BERT and ELMo) which obtain the transferability in an implicit manner (i.e., relying on large-scale corpora), <small>FewNER</small> explicitly optimizes the capability of “learning to adapt quickly” through meta-learning. The results demonstrate that <small>FewNER</small> achieves state-of-the-art performance against nine baseline methods by significant margins on three adaptation experiments (i.e., intra-domain cross-type, cross-domain intra-type and cross-domain cross-type).

BackgroundNatural Language Processing (NLP) applications have developed over the past years in various fields including its application to clinical free text for named entity recognition and relation extraction. However, there has been rapid developments the last few years that there's currently no overview of it. Moreover, it is unclear how these models and tools have been translated into clinical practice. We aim to synthesize and review these developments. MethodsWe reviewed literature from 2010 to date, searching PubMed, Scopus, the Association of Computational Linguistics (ACL), and Association of Computer Machinery (ACM) libraries for studies of NLP systems performing general-purpose (i.e., not disease- or treatment-specific) information extraction and relation extraction tasks in unstructured clinical text (e.g., discharge summaries). ResultsWe included in the review 94 studies with 30 studies published in the last three years. Machine learning methods were used in 68 studies, rule-based in 5 studies, and both in 22 studies. 63 studies focused on Named Entity Recognition, 13 on Relation Extraction and 18 performed both. The most frequently extracted entities were “problem”, “test” and “treatment”. 72 studies used public datasets and 22 studies used proprietary datasets alone. Only 14 studies defined clearly a clinical or information task to be addressed by the system and just three studies reported its use outside the experimental setting. Only 7 studies shared a pre-trained model and only 8 an available software tool. DiscussionMachine learning-based methods have dominated the NLP field on information extraction tasks. More recently, Transformer-based language models are taking the lead and showing the strongest performance. However, these developments are mostly based on a few datasets and generic annotations, with very few real-world use cases. This may raise questions about the generalizability of findings, translation into practice and highlights the need for robust clinical evaluation.

In this paper, an algorithm based on Bi-directional Gated Recurrent Unit (Bi-GRU) and Gradient Boosting Decision Tree (GBDT) is proposed to extract the entity relationship of Traditional Chinese Medicine influenza. Firstly, the word vector is used as the input data set and the word vector is constructed by the word embedding model Word2Vec tool. Then the sentence feature is extracted by Bi-GRU and the attention mechanism is integrated to improve the accuracy of feature extraction. Finally, the feature vector is input into the GBDT algorithm for classification training and prediction to complete the Traditional Chinese Medicine influenza entity relationship extraction. In this paper, a variety of different entity relation extraction algorithms are compared with this algorithm to verify the effectiveness of the algorithm. This algorithm improves the stability of the model and effectively solves the problem of insufficient generalization ability of the model. Therefore, when studying the relationship extraction of Traditional Chinese Medicine texts, we can give priority to using Bi-GRU+GBDT model. Also through the experiment to adjust the model parameters and comparison to get the optimal parameters of Traditional Chinese Medicine influenza relationship extraction experiment.

Relationship extraction is an important task in natural language processing and knowledge mapping. Traditional entity relationship extraction methods have achieved high accuracy in practical applications. However, when faced with the task of entity relationship extraction that is not easy to obtain large-scale supervision training data sets, traditional methods cannot get good results. In this paper, a few-shot relation extraction method based on multi-level feature metric learning is proposed. This method takes the prototype network as the baseline network to generate a class prototype. Firstly, a multi-level feature extraction module is proposed. This module combines the multi-level features of the text with the multi-level attention mechanism, which can fully extract the features of the text. Secondly, a loss function based on label value and negative sample distance is proposed. This algorithm introduces an evaluation mechanism of negative sample distance on the basis of the prototype network, so that the model can adaptively allocate parameters and improve the clustering ability of small samples. Experiments are conducted on FewRel1.0 which is a small sample relational data set. Experiment results show that compared with other models, our model can improve classification accuracy.

In today’s big data era, there are a large number of unstructured information resources on the web. Natural language processing researchers have been working hard to figure out how to extract useful information from them. Entity Relation Extraction is a crucial step in Information Extraction and provides technical support for Knowledge Graphs, Intelligent Q&amp;A systems and Intelligent Retrieval. In this paper, we present a comprehensive history of entity relation extraction and introduce the relation extraction methods based on Machine Mearning, the relation extraction methods based on Deep Learning and the relation extraction methods for open domains. Then we summarize the characteristics and representative results of each type of method and introduce the common datasets and evaluation systems for entity relation extraction. Finally, we summarize current entity relation extraction methods and look forward to future technologies.

Comprehend Medical is a stateless and Health Insurance Portability and Accountability Act (HIPAA) eligible Named Entity Recognition (NER) and Relationship Extraction (RE) service launched under Amazon Web Services (AWS) trained using state-of-the-art deep learning models. Contrary to many existing open source tools, Comprehend Medical is scalable and does not require steep learning curve, dependencies, pipeline configurations, or installations. Currently, Comprehend Medical performs NER in five medical categories: Anatomy, Medical Condition, Medications, Protected Health Information (PHI) and Treatment, Test and Procedure (TTP). Additionally, the service provides relationship extraction for the detected entities as well as contextual information such as negation and temporality in the form of traits. Comprehend Medical provides two Application Programming Interfaces (API): 1) the NERe API which returns all the extracted named entities, their traits and the relationships between them and 2) the PHId API which returns just the protected health information contained in the text. Furthermore, Comprehend Medical is accessible through AWS Console, Java and Python Software Development Kit (SDK), making it easier for non-developers and developers to use.

BackgroundBiodiversity is the assortment of life on earth covering evolutionary, ecological, biological, and social forms. To preserve life in all its variety and richness, it is imperative to monitor the current state of biodiversity and its change over time and to understand the forces driving it. This need has resulted in numerous works being published in this field. With this, a large amount of textual data (publications) and metadata (e.g. dataset description) has been generated. To support the management and analysis of these data, two techniques from computer science are of interest, namely Named Entity Recognition (NER) and Relation Extraction (RE). While the former enables better content discovery and understanding, the latter fosters the analysis by detecting connections between entities and, thus, allows us to draw conclusions and answer relevant domain-specific questions. To automatically predict entities and their relations, machine/deep learning techniques could be used. The training and evaluation of those techniques require labelled corpora.New informationIn this paper, we present two gold-standard corpora for Named Entity Recognition (NER) and Relation Extraction (RE) generated from biodiversity datasets metadata and abstracts that can be used as evaluation benchmarks for the development of new computer-supported tools that require machine learning or deep learning techniques. These corpora are manually labelled and verified by biodiversity experts. In addition, we explain the detailed steps of constructing these datasets. Moreover, we demonstrate the underlying ontology for the classes and relations used to annotate such corpora.