Datasets for generic relation extraction

Two evaluations on Ontology-style relation annotations

Relation Extraction (RE) is at the core of many Natural Language Understanding tasks, including knowledge-base population and Question Answering.However, any Natural Language Processing system is exposed to biases, and the analysis of these has not received much attention in RE.We propose a new method for inspecting bias in the RE pipeline, which is completely transparent in terms of interpretability.Specifically, in this work we analyze biases related to gender and place of birth.Our methodology includes (i) obtaining semantic triplets (subject, object, semantic relation) involving 'person' entities from RE resources, (ii) collecting meta-information ('gender' and 'place of birth') using Entity Linking technologies, and then (iii) analyze the distribution of triplets across different groups (e.g., men versus women).We investigate bias at two levels: In the training data of three commonly used RE datasets (SRED FM , CrossRE, NYT), and in the predictions of a state-of-the-art RE approach (ReLiK).To enable cross-dataset analysis, we introduce a taxonomy of relation types mapping the label sets of different RE datasets to a unified label space.Our findings reveal that bias is a compounded issue affecting underrepresented groups within data and predictions for RE.

Diverse classifiers with label dependencies for long-tail relation extraction in big data

Relation Extraction (RE) is one of the fundamental tasks in Information Extraction. The goal of this task is to find the semantic relations between entity mentions in text. It has been shown in many previous work that the structure of the sentences (i.e., dependency trees) can provide important information/features for the RE models. However, the common limitation of the previous work on RE is the reliance on some external parsers to obtain the syntactic trees for the sentence structures. On the one hand, it is not guaranteed that the independent external parsers can offer the optimal sentence structures for RE and the customized structures for RE might help to further improve the performance. On the other hand, the quality of the external parsers might suffer when applied to different domains, thus also affecting the performance of the RE models on such domains. In order to overcome this issue, we introduce a novel method for RE that simultaneously induces the structures and predicts the relations for the input sentences, thus avoiding the external parsers and potentially leading to better sentence structures for RE. Our general strategy to learn the RE-specific structures is to apply two different methods to infer the structures for the input sentences (i.e., two views). We then introduce several mechanisms to encourage the structure and semantic consistencies between these two views so the effective structure and semantic representations for RE can emerge. We perform extensive experiments on the ACE 2005 and SemEval 2010 datasets to demonstrate the advantages of the proposed method, leading to the state-of-the-art performance on such datasets.

In the expanding landscape of biomedical literature, numerous latent associations outlined in scholarly papers await discovery and integration into biomedical databases. Biomedical Natural Language Processing (NLP) research focuses on automating knowledge extraction and mining from this literature, particularly emphasizing the essential task of Relation Extraction (RE). However, existing models have limitations, mainly in their applicability to partial datasets for RE tasks. Moreover, conventional models often treat RE as a binary classification challenge, which proves suboptimal given the diverse relationships, including intricate ones like similarity and hierarchy, present in the RE dataset. These limitations are exacerbated by the models’ inability to capture word-level positional nuances and sentence-level language features. In response to these challenges, we present a novel RE model called BicapBert. This model integrates neural networks and capsule networks, enhancing them with hybrid knowledge graph embeddings to extract relevant features. BicapBert utilizes PubMedBERT and capsule networks to extract word-level positional and sentence-level language features. It further captures knowledge features from a biomedical knowledge graph, integrating them with the aforementioned linguistic features. The amalgamated information is then input into a multi-layer perceptron, culminating in results derived through a softmax classifier. Experimental evaluations on three extensive RE task datasets showcase the state-of-the-art performance of our proposed model. Additionally, we validate the model’s efficacy on three randomly selected biomedical datasets for various tasks, further affirming its superiority.

Relation Extraction is the task of identifying entity mention spans in raw text and then identifying relations between pairs of the entity mentions. Recent approaches for this span-level task have been token-level models which have inherent limitations. They cannot easily define and implement span-level features, cannot model overlapping entity mentions and have cascading errors due to the use of sequential decoding. To address these concerns, we present a model which directly models all possible spans and performs joint entity mention detection and relation extraction. We report a new state-of-the-art performance of 62.83 F1 (prev best was 60.49) on the ACE2005 dataset.

Document-level relation extraction (RE), which requires integrating and reasoning information to identify multiple possible relations among entities. However, previous research typically performed reasoning on heterogeneous graphs and set a global threshold for multiple relations classification, regardless of interaction reasoning information among multiple relations and positive–negative samples imbalance on databases. This paper proposes a novel framework for Document-level RE with two techniques, dual attention fusion and dynamic asymmetric loss. Concretely, to obtain more interdependency feature learning, we construct entity pairs and contextual matrixes using multi-head axial attention and co-attention mechanism to learn the interaction among entity pairs deeply. To alleviate the hard-thresholds influence from positive–negative imbalance samples, we dynamically adjust weights to optimize the probabilities of different labels. We evaluate our model on two benchmark document-level RE datasets, DocRED and CDR. Experimental results show that our DASL (Dual Attention fusion and dynamic aSymmetric Loss) obtains superior performance on two public datasets, we further provide extensive experiments to analyze how dual attention fusion and dynamic asymmetric loss guide the model for better extracting multi-label relations among entities.

Document-level relation extraction (RE) has attracted considerable attention, because a large number of relational facts are expressed in multiple sentences. Recently, encoder-aggregator based models have become promising for document-level RE. However, these models have two shortcomings: (i) they cannot obtain contextualized representations of a mention by low computational cost, when the mention is involved in different entity pairs; (ii) they ignore the different weights for the mention pairs of a target entity pair. To tackle the above two problems, in this paper, we propose a novel encoder-attender-aggregator model, which introduces two attenders between the encoder and aggregator. Specifically, a mutual attender is first employed on the selected head and tail mentions to efficiently produce contextualized mention representations. Then, an integration attender is utilized to weight the mention pairs of a target entity pair. Extensive experiments on two document-level RE datasets show that the proposed model performs better than the state-of-the-art baselines. Our codes are publicly available at "https://github.com/nefujiangping/EncAttAgg".

Document-level relation extraction (RE) poses new challenges compared to its sentence-level counterpart. One document commonly contains multiple entity pairs, and one entity pair occurs multiple times in the document associated with multiple possible relations. In this paper, we propose two novel techniques, adaptive thresholding and localized context pooling, to solve the multi-label and multi-entity problems. The adaptive thresholding replaces the global threshold for multi-label classification in the prior work with a learnable entities-dependent threshold. The localized context pooling directly transfers attention from pre-trained language models to locate relevant context that is useful to decide the relation. We experiment on three document-level RE benchmark datasets: DocRED, a recently released large-scale RE dataset, and two datasets CDRand GDA in the biomedical domain. Our ATLOP (Adaptive Thresholding and Localized cOntext Pooling) model achieves an F1 score of 63.4, and also significantly outperforms existing models on both CDR and GDA. We have released our code at https://github.com/wzhouad/ATLOP.

Relation extraction is a natural language processing (NLP) task to extract semantic relations between entities from unstructured texts, and is an important research content of constructing knowledge graph. Based on the documents issued by the China Banking and Insurance Regulatory Commission, we construct a knowledge graph in the field of financial regulation through relation extraction, which helps bank staff to realize intelligent query of regulatory documents and locate business-related regulatory requirements quickly and accurately. This paper proposes the Financial Regulation BERT (FR-BERT) model for relation extraction in the financial regulation field, aiming at the dataset's characteristics of rich semantic information of target entities, clear keywords and long texts. FR-BERT uses the BERT model to obtain sentence vector information containing each word vector, then obtains the target entity vectors and keyword vector by locating the sentence vector information, and obtains the text vector by sending the whole sentence vector information into Bi-directional Long Short-Term Memory (BiLSTM). Finally, the model uses the above vectors for classification information. FR-BERT integrates target entity vectors, keyword vector, and text vector output by BiLSTM to achieve relation extraction between entities. Compared with other relation extraction models selected in this paper, the experimental results show that FR-BERT performs better such as Macro-F1 on the relation extraction task in the financial regulation field, which verifies the effectiveness of the method.

Relation extraction from the Web data has attracted a lot of attention recently. However, little work has been done when it comes to the enterprise data regardless of the urgent needs to such work in real applications (e.g., E-discovery). One distinct characteristic of the enterprise data (in comparison with the Web data) is its low redundancy. Previous work on relation extraction from the Web data largely relies on the data's high redundancy level and thus cannot be applied to the enterprise data effectively. This chapter reviews related work on relation extraction and introduces an unsupervised hybrid framework REACTOR for semantic relation extraction over enterprise data. REACTOR combines a statistical method, classification, and clustering to identify various types of relations among entities appearing in the enterprise data automatically. REACTOR was evaluated over a real-world enterprise data set from HP that contains over three million pages and the experimental results show its effectiveness.

Named entity extraction (NEE) and disambiguation (NED) are two areas of research that are well covered in literature. Typical fields addressing these topics are information retrieval, natural language processing, and semantic web. Although these topics are highly dependent, almost no existing works examine this dependency. It is the aim of this position paper to explore that dependency and show how one affects the other, and vice versa. We show the benefit of using this reinforcement effect on two domains: NEE and NED for toponyms in formal text; and for arbitrary entity types in informal short text in tweets. Finally we give an insight about the potential of this approach for future research.

DWIE: An entity-centric dataset for multi-task document-level information extraction

Entity relation extraction plays an important role in the biomedical, healthcare, and clinical research areas. Recently, pre-trained models based on transformer architectures and their variants have shown remarkable performances in various natural language processing tasks. Most of these variants were based on slight modifications in the architectural components, representation schemes and augmenting data using distant supervision methods. In distantly supervised methods, one of the main challenges is pruning out noisy samples. A similar situation can arise when the training samples are not directly available but need to be constructed from the given dataset. The BioCreative V Chemical Disease Relation (CDR) task provides a dataset that does not explicitly offer mention-level gold annotations and hence replicates the above scenario. Selecting the representative sentences from the given abstract or document text that could convey a potential entity relationship becomes essential. Most of the existing methods in literature propose to either consider the entire text or all the sentences which contain the entity mentions. This could be a computationally expensive and time consuming approach. This paper presents a novel approach to handle such scenarios, specifically in biomedical relation extraction. We propose utilizing the Shortest Dependency Path (SDP) features for constructing data samples by pruning out noisy information and selecting the most representative samples for model learning. We also utilize triplet information in model learning using the biomedical variant of BERT, viz., BioBERT. The problem is represented as a sentence pair classification task using the sentence and the entity-relation pair as input. We analyze the approach on both intra-sentential and inter-sentential relations in the CDR dataset. The proposed approach that utilizes the SDP and triplet features presents promising results, specifically on the inter-sentential relation extraction task. We make the code used for this work publicly available on Github.11https://github.com/vanikanjirangat/BioRelation-Extraction_BERT_SDP_Triplets

Most existing relation extraction approaches exclusively target binary relations, and n-ary relation extraction is relatively unexplored. Current state-of-the-art n-ary relation extraction method is based on a supervised learning approach and, therefore, may suffer from the lack of sufficient relation labels. In this paper, we propose a novel approach to cross-sentence n-ary relation extraction based on universal schemas. To alleviate the sparsity problem and to leverage inherent decomposability of n-ary relations, we propose to learn relation representations of lower-arity facts that result from decomposing higher-arity facts. The proposed method computes a score of a new n-ary fact by aggregating scores of its decomposed lower-arity facts. We conduct experiments with datasets for ternary relation extraction and empirically show that our method improves the n-ary relation extraction performance compared to previous methods.