Biomedical named entity recognition using deep neural networks with contextual information

Character level and word level embedding with bidirectional LSTM – Dynamic recurrent neural network for biomedical named entity recognition from literature

Weaponry equipment names belong to an important military naming entity that is difficult to identify because of features, such as complex components, miscellaneous, and scarce annotation corpus. Here, the automatic recognition of weaponry equipment names is specifically explored, a NER (Named Entity Recognition) algorithm is proposed based on BI-LSTM-CRF (Bi-directional Long Short Term Memory Conditional Random Field), thereby demonstrating the effectiveness of domain features in domain-specific entity recognition. Firstly, Chinese characters are represented by word embedding and input into the model. Then, the input feature vector sequence is processed by BI-LSTM (Bi-directional Long Short Term Memory) NN (Neural Network) to extract context semantic learning features. Finally, the learned features are connected to the linear CRF (Conditional Random Field), the NEs (Named Entities) in the equipment support field are labeled, and the NER results are obtained and output. The experimental results show that the accuracy of the NER algorithm based on the BI-LSTM-CRF model is 92.02%, the recall rate is 93.21%, and the F1 value reaches 93.88%. The effect of this model is better than the BI-LSTM NN model and LSTM-CRF (Long Short Term Memory Conditional Random Field) NN model. The proposed model provides some references for entity recognition in the field of equipment support.

The rapid advancement of artificial intelligence and intelligent algorithms has led to the widespread adoption of natural language processing technology. Named entity recognition (NER), an important task in this field, takes a pivotal part in information extraction, machine translation, and knowledge graph construction. However, for a small language such as Lao, traditional methods have obvious deficiencies in recognition effectiveness due to its complex syllable structure, fuzzy word boundaries and scarce corpus resources. To address this problem, the study designs a named entity recognition model for Lao that incorporates a Bi-directional Long Short-Term Memory (BiLSTM) network, an attention mechanism and a conditional random field. The model first vectorizes multi-granularity features such as characters, syllables and lexemes through an embedding layer, then captures contextual dependencies by a BiLSTM network, introduces an Attention to highlight key information, and finally establishes a global label transfer relation through a conditional random field to ensure the consistency of the predicted sequence. The findings denote that the model achieves 91.5%, 91.0%, 90.7%, and 90.9% in accuracy rate, precision rate, recall rate, and F1 score, respectively, and the mean absolute error and mean square error are reduced to 0.062 and 0.011, which are significantly better than the comparison model. The results show that combining multi-level feature modeling and global constraint mechanism can effectively enhance the performance of named entity recognition in Lao, which provides a valuable reference and practical path for natural language processing in small languages.

Automatic Parts of Speech(POS) tagging is one of the most fundamental tasks for a language in Natural Language Processing(NLP), which acts as a feature for solving advanced NLP tasks. Named Entity Recognition(NER) is another essential task of NLP for information retrieval. Researchers could not find up to the mark solution yet on these two tasks for Bangla language compared to other languages, for instance, English, Ger-man. Moreover, many solutions heavily depend on handcrafted features that require strong linguistic expertise. As these two sequence labeling tasks are similar, In this work, two different datasets of POS tagging and NER were prepared, and different deep neural network approaches studied for solving these two tasks separately. All of the approaches were end to end and did not need any handcrafted feature like word suffixes or affixes, gazetteers, dictionary. This study came up with an end to end solution using deep neural network-based model consisting of Bi-directional Long short-term memory(BLSTM), Convolutional Neural Network(CNN) and Conditional Random Field(CRF). The proposed model trained on respected datasets achieved an accuracy of 93.86% on POS tagging and a strict f1 score of 0.6285 on NER on prepared datasets, respectively.

In this paper, the named entity recognition (NER) technology is applied to the field of Qu Tan temple murals to identify religious entities in the text. In order to obtain a machine-readable corpus, the relevant books are scanned and optical character recognition is performed, and then the data is manually labeled to form the named entity recognition dataset needed in this study. In this paper, the bidirectional encoder representations from transformers (BERT) model is used to encode the input data, and the obtained word vector can effectively extract the features of the text by making use of the relationship between words. In order to obtain more accurate context information, the bidirectional long short term memory (BiLSTM) is used to extract features and output the prediction label corresponding to each word. After the BiLSTM model, the conditional random field (CRF) algorithm is joined, which will correct the score of the output, and the output is the optimal label sequence. In this study, the performance of the model is also improved by data enhancement.

In the rapidly evolving field of cybersecurity, the integration of multi-source, heterogeneous, and fragmented data into a coherent knowledge graph has garnered considerable attention. Such a graph elucidates semantic interconnections, thereby facilitating sophisticated analytical decision support. Central to the construction of a cybersecurity knowledge graph is Named Entity Recognition (NER), a critical technology that converts unstructured text into structured data. The efficacy of NER is pivotal, as it directly influences the integrity of the knowledge graph. The task of NER in cybersecurity, particularly within the Chinese linguistic context, presents distinct challenges. Chinese text lacks explicit space delimiters and features complex contextual dependencies, exacerbating the difficulty in discerning and categorizing named entities. These linguistic characteristics contribute to errors in word segmentation and semantic ambiguities, impeding NER accuracy. This paper introduces a novel NER methodology tailored for the Chinese cybersecurity corpus, termed CSBERT-IDCNN-BiLSTM-CRF. This approach harnesses Iterative Dilated Convolutional Neural Networks (IDCNN) for extracting local features, and Bi-directional Long Short-Term Memory networks (BiLSTM) for contextual understanding. It incorporates CSBERT, a pre-trained model adept at processing few-shot data, to derive input feature representations. The process culminates with Conditional Random Fields (CRF) for precise sequence labeling. To compensate for the scarcity of publicly accessible Chinese cybersecurity datasets, this paper synthesizes a bespoke dataset, authenticated by data from the China National Vulnerability Database, processed via the YEDDA annotation tool. Empirical analysis affirms that the proposed CSBERT-IDCNN-BiLSTM-CRF model surpasses existing Chinese NER frameworks, with an F1-score of 87.30% and a precision rate of 85.89%. This marks a significant advancement in the accurate identification of cybersecurity entities in Chinese text, reflecting the model’s robust capability to address the unique challenges presented by the language’s structural intricacies.

Biomedical named entity recognition (BioNER) is one subtask of named entity recognition (NER) research. Although there are a number of named entity recognition systems, they can not obtain good performances extended to biomedical subfield. BioNER becomes a challenging work. We employ a skip-chain conditional random fields (CRFs) model for BioNER. This model completely considers to the long-range dependencies about biomedical information. These distant dependencies are powerful to identify some frequent appearing named entities and to classify them, especially for both classes protein and cell type. When we test the GENIA corpus, our approach obtains significant improvement over other methods, which achieves precision, recall and F-score of 72.8%, 73.6% and 73.2%, respectively.

BackgroundDisease named entity recognition (NER) is a fundamental step in information processing of medical texts. However, disease NER involves complex issues such as descriptive modifiers in actual practice. The accurate identification of disease NER is a still an open and essential research problem in medical information extraction and text mining tasks.MethodsA hybrid model named Semantics Bidirectional LSTM and CRF (SBLC) for disease named entity recognition task is proposed. The model leverages word embeddings, Bidirectional Long Short Term Memory networks and Conditional Random Fields. A publically available NCBI disease dataset is applied to evaluate the model through comparing with nine state-of-the-art baseline methods including cTAKES, MetaMap, DNorm, C-Bi-LSTM-CRF, TaggerOne and DNER.ResultsThe results show that the SBLC model achieves an F1 score of 0.862 and outperforms the other methods. In addition, the model does not rely on external domain dictionaries, thus it can be more conveniently applied in many aspects of medical text processing.ConclusionsAccording to performance comparison, the proposed SBLC model achieved the best performance, demonstrating its effectiveness in disease named entity recognition.

Named Entity Recognition (NER) is a vital task in natural language processing (NLP) that involves identifying and classifying named entities in text. This paper focuses on the comparison of four different algorithms of NER techniques in the context of India. The four different algorithms are CRF (Conditional Random Fields), Stanza, Spacy, and Nameparser. The accuracy will be compared with the expected accuracy of all four methods of NER. Accuracy will be obtained after running the method on a dataset or text. To calculate the accuracy of four algorithms a large dataset of Indian names will be used. A comparison graph was obtained to display the accuracy graph of all four algorithms. "NLP-Named Entity Recognition in India" which focuses on detecting person names, states, and districts, and suggesting names based on religion. Named Entity Recognition (NER) is a Natural Language Processing (NLP) task that identifies and classifies named entities in text. Named entities are typically proper nouns, such as person names, organization names, and location names. NER is a challenging task for Indian languages, due to the diversity of Indian languages and the lack of resources for training NER systems. The proposed system can be used to improve the accuracy and efficiency of a variety of NLP applications in India.

Named entity recognition (NER) is one of the preprocessing stages in natural language processing (NLP), which functions to detect and classify entities in the corpus. NER results are used in various NLP applications, including sentiment analysis, text summarization, chatbot, machine translation, and question answering. Several previous reviews partially discussed NER, for instance, NER reviews in specific domains, NER classification, and NER deep learning. This paper provides a comprehensive and systematic review on NER topic studies published from 2011 to 2020. The main contribution of this review is to present a comprehensive systematic literature review on NER from preprocessing techniques, datasets, application domains, feature extraction techniques, approaches, methods, and evaluation techniques. The result concludes that the deep learning approach and the Bi-directional long short-term memory with a conditional random field (Bi-LSTM-CRF) method are the most interesting methods among NER researchers. At the same time, medical and health are NER researchers' most popular domains. These developments have also led to an increasing number of public datasets in the medical and health fields. At the end of this review, we recommend some opportunities and challenges for NER research going forward.

In the era of information explosion, named entity recognition (NER) has attracted widespread attention in the field of natural language processing, as it is fundamental to information extraction. Recently, methods of NER based on representation learning, e.g., character embedding and word embedding, have demonstrated promising recognition results. However, existing models only consider partial features derived from words or characters while failing to integrate semantic and syntactic information, e.g., capitalization, inter-word relations, keywords, and lexical phrases, from multilevel perspectives. Intuitively, multilevel features can be helpful when recognizing named entities from complex sentences. In this study, we propose a novel attentive multilevel feature fusion (AMFF) model for NER, which captures the multilevel features in the current context from various perspectives. It consists of four components to, respectively, capture the local character-level (CL), global character-level (CG), local word-level (WL), and global word-level (WG) features in the current context. In addition, we further define document-level features crafted from other sentences to enhance the representation learning of the current context. To this end, we introduce a novel context-aware attentive multilevel feature fusion (CAMFF) model based on AMFF, to fully leverage document-level features from all the previous inputs. The obtained multilevel features are then fused and fed into a bidirectional long short-term memory (BiLSTM)-conditional random field (CRF) network for the final sequence labeling. Extensive experiments on four benchmark datasets demonstrate that our proposed AMFF and CAMFF models outperform a set of state-of-the-art baseline methods and the features learned from multiple levels are complementary.

In the field of Natural Language Processing, one of the very important research areas of Information Extraction (IE) comes in Named Entity Recognition (NER). NER is a subtask of IE that seeks to identify and classify the predefined categories of named entities in text documents. Considerable amount of work has been done on NER in recent years due to the increasing demand of automated texts and the wide availability of electronic corpora. While it is relatively easy and natural for a human reader to read and understand the context of a given article, getting a machine to understand and differentiate between words is a big challenge. For instance, the word ‘brown’ may refer to a person called Mr. Brown, or the colour of an item which is brown. Human readers can easily discern the meaning of the word by looking at the context of that particular sentence, but it would be almost impossible for a computer to interpret it without any additional information. To deal with the issue, researchers in NER field have proposed various rule-based systems (Wakao, Gaizauskas & Wilks, 1996; Krupka & Hausman, 1998; Maynard, Tablan, Ursu, Cunningham & Wilks, 2001). These systems are able to achieve high accuracy in recognition with the help of some lists of known named entities called gazetteers. The problem with rule-based approach is that it lacks the robustness and portability. It incurs steep maintenance cost especially when new rules need to be introduced for some new information or new domains. A better option is thus to use machine learning approach that is trainable and adaptable. Three wellknown machine learning approaches that have been used extensively in NER are Hidden Markov Model (HMM), Maximum Entropy Model (MEM) and Decision Tree. Many of the existing machine learning-based NER systems (Bikel, Schwartz & Weischedel, 1999; Zhou & Su, 2002; Borthwick, Sterling, Agichten & Grisham, 1998; Bender, Och & Ney, 2003; Chieu & Ng, 2002; Sekine, Grisham & Shinnou, 1998) are able to achieve near-human performance for named entity tagging, even though the overall performance is still about 2% short from the rule-based systems. There have also been many attempts to improve the performance of NER using a hybrid approach with the combination of handcrafted rules and statistical models (Mikheev, Moens & Grover, 1999; Srihari & Li, 2000; Seon, Ko, Kim & Seo, 2001). These systems can achieve relatively good performance in the targeted domains owing to the comprehensive handcrafted rules. Nevertheless, the portability problem still remains unsolved when it comes to dealing with NER in various domains. As such, this article presents a hybrid machine learning approach using MEM and HMM successively. The reason for using two statistical models in succession instead of one is due to the distinctive nature of the two models. HMM is able to achieve better performance than any other statistical models, and is generally regarded as the most successful one in machine learning approach. However, it suffers from sparseness problem, which means considerable amount of data is needed for it to achieve acceptable performance. On the other hand, MEM is able to maintain reasonable performance even when there is little data available for training purpose. The idea is therefore to walkthrough the testing corpus using MEM first in order to generate a temporary tagging result, while this procedure can be simultaneously used as a training process for HMM. During the second walkthrough, the corpus uses HMM for the final tagging. In this process, the temporary tagging result generated by MEM will be used as a reference for subsequent error checking and correction. In the case when there is little training data available, the final result can still be reliable based on the contribution of the initial MEM tagging result.

Biomedical named entity recognition (Bio-NER) is one of the most fundamental tasks in the field of biomedical information extraction. The accuracy of biomedical named entity recognition is crucial to the follow-up research work. This paper presents a method for named entity recognition based on the concept of three-way decisions. The method uses a discriminative approach named conditional random fields (CRFs) to construct models. These models follow the decision-making rule of three-way decision in all stages, the model cannot make decision arbitrarily when the information is incomplete until it gets more information. The experimental results show that our method can improve the performance for biomedical named entity recognition compared with other methods.

Chinese pesticide named-entity recognition (NER) aims to identify named entities related to pesticide properties from unstructured Chinese pesticide information texts. In view of the characteristics of massive, fragmented, professional, and complex semantic relationships of pesticide information data, a deep learning method based on multi-feature fusion was applied to improve the accuracy of pesticide NER. In this study, the pesticide data set is manually annotated by the begin inside outside (BIO) sequence annotation scheme. Bi-directional long short-term memory (BiLSTM) and iterated dilated convolutional neural networks (IDCNN) combined with conditional random field (CRF) form the model BiLSTM-IDCNN-CRF, and it is applied to implement named-entity recognition in Chinese pesticide data sets. IDCNN is introduced to enhance the semantic representation ability and local feature capture ability of the text. BiLSTM network and IDCNN network are combined to obtain the long-distance dependence relationship and context features of different granularity of pesticide data text. Finally, CRF is used to implement the sequence labeling task. According to the experiment results, the accuracy rate, recall rate, and F1 score of the BiLSTM-IDCNN-CRF model in the Chinese pesticide data set were 78.59%, 68.71%, and 73.32%, respectively, which are significantly better than other compared models. Experiments show that the BiLSTM-IDCNN-CRF model can effectively identify and extract entities from Chinese pesticide information text data, which is helpful in constructing the pesticide information knowledge graph and intelligent question-answering.

Information extraction (IE) is a textual information processing task concerned with the automatic extraction of entity mentions and relational structures from documents, which has been widely studied and applied in various fields such as biomedicine and news. In this paper, we extend this technique to biographies and propose a new BioIE3 dataset to extract the authors' experiences (including birth, education, work and awards) from their profiles in the scientific literatures. Named entity recognition (NER) and relation extraction (RE) tasks are performed simultaneously in a neural architecture based on the Skip-Gram Word2vec, Bidirectional Long Short Term Memory (BiLSTM) and Conditional Random Fields (CRF) models. On the basis of the above work, negative examples are especially introduced into the training stage to effectively improve and increase the robustness of the model.