Improving Document-level Relation Extraction via Contextualizing Mention Representations andWeighting Mention Pairs

Document-level relation extraction (RE), which requires integrating and reasoning information to identify multiple possible relations among entities. However, previous research typically performed reasoning on heterogeneous graphs and set a global threshold for multiple relations classification, regardless of interaction reasoning information among multiple relations and positive–negative samples imbalance on databases. This paper proposes a novel framework for Document-level RE with two techniques, dual attention fusion and dynamic asymmetric loss. Concretely, to obtain more interdependency feature learning, we construct entity pairs and contextual matrixes using multi-head axial attention and co-attention mechanism to learn the interaction among entity pairs deeply. To alleviate the hard-thresholds influence from positive–negative imbalance samples, we dynamically adjust weights to optimize the probabilities of different labels. We evaluate our model on two benchmark document-level RE datasets, DocRED and CDR. Experimental results show that our DASL (Dual Attention fusion and dynamic aSymmetric Loss) obtains superior performance on two public datasets, we further provide extensive experiments to analyze how dual attention fusion and dynamic asymmetric loss guide the model for better extracting multi-label relations among entities.

Document-level relation extraction (RE) poses new challenges compared to its sentence-level counterpart. One document commonly contains multiple entity pairs, and one entity pair occurs multiple times in the document associated with multiple possible relations. In this paper, we propose two novel techniques, adaptive thresholding and localized context pooling, to solve the multi-label and multi-entity problems. The adaptive thresholding replaces the global threshold for multi-label classification in the prior work with a learnable entities-dependent threshold. The localized context pooling directly transfers attention from pre-trained language models to locate relevant context that is useful to decide the relation. We experiment on three document-level RE benchmark datasets: DocRED, a recently released large-scale RE dataset, and two datasets CDRand GDA in the biomedical domain. Our ATLOP (Adaptive Thresholding and Localized cOntext Pooling) model achieves an F1 score of 63.4, and also significantly outperforms existing models on both CDR and GDA. We have released our code at https://github.com/wzhouad/ATLOP.

In document-level relation extraction (DocRE), graph structure is generally used to encode relation information in the input document to classify the relation category between each entity pair, and has greatly advanced the DocRE task over the past several years. However, the learned graph representation universally models relation information between all entity pairs regardless of whether there are relationships between these entity pairs. Thus, those entity pairs without relationships disperse the attention of the encoder-classifier DocRE for ones with relationships, which may further hind the improvement of DocRE. To alleviate this issue, we propose a novel encoder-classifier-reconstructor model for DocRE. The reconstructor manages to reconstruct the ground-truth path dependencies from the graph representation, to ensure that the proposed DocRE model pays more attention to encode entity pairs with relationships in the training. Furthermore, the reconstructor is regarded as a relationship indicator to assist relation classification in the inference, which can further improve the performance of DocRE model. Experimental results on a large-scale DocRE dataset show that the proposed model can significantly improve the accuracy of relation extraction on a strong heterogeneous graph-based baseline. The code is publicly available at https://github.com/xwjim/DocRE-Rec.

This paper aims to study how to use remote monitoring for relation extraction, so as to identify the relationship between entities and provide logical facts. In recent years, document-level relation extraction (DocRE) has attracted much attention. Distant supervision (DS) has been playing an important role in sentence-level relation extraction (RE). Compared with traditional relation extraction methods, DS requires less data. However, the performance of document-level DS is limited by the noise in the long distant text. Previous DS works represent intra-sentential and inter-sentential representations in the same way, and neglect the difference between noise in them. This paper proposes an architecture called RDRS, which represents inter-sentence and intra-sentence relations in different ways, and uses a sentence-level relation model to represent document-level relations. The RDRS contains various modules, in which pairs of entities are separated according to the distribution of their mentions. In addition, this paper also improves the performance of inter-sentence representation by associating intra-sentence representation with inter-sentence representation. The experiments are carried out on Document-level Relation Extraction Dataset (DocRED). Experiments on DocRED show that the method has a significant improvement over the baseline. This model is better than previous models that represent inter-sentence and intra-sentence relations in the same way.

DDI is an important part of drug-related research and pharmacovigilance. Extracting DDI information from scientific literature has become a low-cost and highly reliable way. Currently, existing works are all sentence-level DDI relation extraction. In fact, the entity relationship is often expressed by multiple sentences. Moreover, the sentence-level DDI relation extraction also causes a large amount of redundancy in the whole dataset with increasing in negative instance data. In this study, we propose a document-level DDI relation extraction method based on document-entity embedding. Our method performs special processing on the DDI Extraction 2013 for the first time, in order to calculate document-level relation extraction. For obtaining document-level entity information, we propose a document-entity embedding method to integrate the information of all same drugs in the same article. The experimental results show that the processing of DDI Extraction 2013 dataset is reasonable. In addition, the proposed method has achieved good performance on document-level DDI dataset, and the best F1 score is 62.51%. This is the first time that DDI Extraction 2013 has been processed into a document-level dataset, and document-level DDI relation extraction has been realized.

Document-level relation extraction aims to identify relationships between entities across a document and plays a vital role in downstream tasks such as knowledge graph construction. However, most existing methods assume that entities and their coreferences are pre-annotated. This assumption can lead to error propagation and limit error correction, thereby degrading real-world performance. While recent approaches that integrate graph neural networks with Transformer-based models have shown promising results, the interaction between document encoders and graph encoders is often unidirectional and insufficient. Moreover, graph encoders frequently struggle to capture the global context of entity nodes, hindering their ability to model cross-sentence reasoning. To tackle these challenges, we propose a Graph-Transformer-based joint relation extraction model that employs a multi-graph joint modeling framework, including an entity aggregation graph, a semantic relation graph, and a global structural graph, to comprehensively represent semantic and structural associations among entities. We enhance a multi-layer relational graph convolutional network (R-GCN) with structure-aware self-attention and graph-text cross-attention. These mechanisms help better fuse structural and semantic information. Furthermore, we introduce a Graph-to-Text Adversarial Distillation (GTAD) mechanism to effectively transfer structural knowledge from the graph encoder to the document encoder. The experimental results demonstrate that the proposed method achieves superior performance across multiple document-level relation extraction tasks.

Document-level relation extraction with entity mentions deep attention

Document-level relation extraction with multi-layer heterogeneous graph attention network

Document-level relation extraction (DocRE) aims to extract relations among entities within a document, which is crucial for applications like knowledge graph construction. Existing methods usually assume that entities and their mentions are identified beforehand, which falls short of real-world applications. To overcome this limitation, we propose TaG, a novel table-to-graph generation model for joint extractionof entities and relations at document-level. To enhance the learning of task dependencies, TaG induces a latent graph among mentions, with different types of edges indicating different task information, which is further broadcast with a relational graph convolutional network. To alleviate the error propagation problem, we adapt the hierarchical agglomerative clustering algorithm to back-propagate task information at decoding stage. Experiments on the benchmark dataset, DocRED, demonstrate that TaG surpasses previous methods by a large margin and achieves state-of-the-art results.

Relation extraction (RE) is the core link of downstream tasks, such as information retrieval, question answering systems, and knowledge graphs. Most of the current mainstream RE technologies focus on the sentence-level corpus, which has great limitations in practical applications. Moreover, the previously proposed models based on graph neural networks or transformers try to obtain context features from the global text, ignoring the importance of local features. In practice, the relation between entity pairs can usually be inferred just through a few keywords. This paper proposes a keyword detection and context filtering method based on the Self-Attention mechanism for document-level RE. In addition, a Self-Attention Memory (SAM) module in ConvLSTM is introduced to process the document context and capture keyword features. By searching for word embeddings with high cross-attention of entity pairs, we update and record critical local features to enhance the performance of the final classification model. The experimental results on three benchmark datasets (DocRED, CDR, and GBA) show that our model achieves advanced performance within open and specialized domain relationship extraction tasks, with up to 0.87% F1 value improvement compared to the state-of-the-art methods. We have also designed experiments to demonstrate that our model can achieve superior results by its stronger contextual filtering capability compared to other methods.

Relation Extraction (RE) is to obtain a predefined relation type of two entities mentioned in a piece of text, e.g., a sentence-level or a document-level text. Most existing studies suffer from the noise in the text, and necessary pruning is of great importance. The conventional sentence-level RE task addresses this issue by a denoising method using the shortest dependency path to build a long-range semantic dependency between entity pairs. However, this kind of denoising method is scarce in document-level RE. In this work, we explicitly model a denoised document-level graph based on linguistic knowledge to capture various long-range semantic dependencies among entities. We first formalize a Syntactic Dependency Tree forest (SDT-forest) by introducing the syntax and discourse dependency relation. Then, the Steiner tree algorithm extracts a mention-level denoised graph, Steiner Graph (SG), removing linguistically irrelevant words from the SDT-forest. We then devise a slide residual attention to highlight word-level evidence on text and SG. Finally, the classification is established on the SG to infer the relations of entity pairs. We conduct extensive experiments on three public datasets. The results evidence that our method is beneficial to establish long-range semantic dependency and can improve the classification performance with longer texts.

Distant supervision (DS) has been widely used to generate auto-labeled data for sentence-level relation extraction (RE), which improves RE performance. However, the existing success of DS cannot be directly transferred to the more challenging document-level relation extraction (DocRE), since the inherent noise in DS may be even multiplied in document level and significantly harm the performance of RE. To address this challenge, we propose a novel pre-trained model for DocRE, which denoises the document-level DS data via multiple pre-training tasks. Experimental results on the large-scale DocRE benchmark show that our model can capture useful information from noisy DS data and achieve promising results.

Document-level relation extraction aims to extract relations among entities within a document. Different from sentence-level relation extraction, it requires reasoning over multiple sentences across paragraphs. In this paper, we propose Graph Aggregation-and-Inference Network (GAIN), a method to recognize such relations for long paragraphs. GAIN constructs two graphs, a heterogeneous mention-level graph (MG) and an entity-level graph (EG). The former captures complex interaction among different mentions and the latter aggregates mentions underlying for the same entities. Based on the graphs we propose a novel path reasoning mechanism to infer relations between entities. Experiments on the public dataset, DocRED, show GAIN achieves a significant performance improvement (2.85 on F1) over the previous state-of-the-art. Our code is available at https://github.com/PKUnlp-icler/GAIN.

Document-level relation extraction aims to extract the relationship among the entities in a paragraph of text. Compared with sentence-level, the text in document-level relation extraction is much longer and contains many more entities. It makes the document-level relation extraction a harder task. The number and complexity of entities make it necessary to provide enough information about the entities for the models in document-level relation extraction. To solve this problem, we put forward a document-level entity mask method with type information (DEMMT), which masks each mention of the entities by special tokens. By using this entity mask method, the model can accurately obtain every mention and type of the entities. Based on DEMMT, we propose a BERT-based one-pass model, through which we can predict the relationships among the entities by processing the text once. We test the proposed model on the DocRED dataset, which is a large scale open-domain document-level relation extraction dataset. The results on the manually annotated part of DocRED show that our approach obtains 6% F1 improvement compared with the state-of-the-art models that do not use pre-trained models and has 2% F1 improvement than BERT which does not use the DEMMT. On the distant supervision generated part of DocRED, the improvement of F1 is 2% compared with no pre-trained models, and 5% compared with pure BERT.

Relation Extraction (RE) is at the core of many Natural Language Understanding tasks, including knowledge-base population and Question Answering.However, any Natural Language Processing system is exposed to biases, and the analysis of these has not received much attention in RE.We propose a new method for inspecting bias in the RE pipeline, which is completely transparent in terms of interpretability.Specifically, in this work we analyze biases related to gender and place of birth.Our methodology includes (i) obtaining semantic triplets (subject, object, semantic relation) involving 'person' entities from RE resources, (ii) collecting meta-information ('gender' and 'place of birth') using Entity Linking technologies, and then (iii) analyze the distribution of triplets across different groups (e.g., men versus women).We investigate bias at two levels: In the training data of three commonly used RE datasets (SRED FM , CrossRE, NYT), and in the predictions of a state-of-the-art RE approach (ReLiK).To enable cross-dataset analysis, we introduce a taxonomy of relation types mapping the label sets of different RE datasets to a unified label space.Our findings reveal that bias is a compounded issue affecting underrepresented groups within data and predictions for RE.