Named Entity Recognition in Equipment Support Field Using Tri-Training Algorithm and Text Information Extraction Technology

BackgroundIn biomedical text mining, named entity recognition (NER) is an important task used to extract information from biomedical articles. Previously proposed methods for NER are dictionary- or rule-based methods and machine learning approaches. However, these traditional approaches are heavily reliant on large-scale dictionaries, target-specific rules, or well-constructed corpora. These methods to NER have been superseded by the deep learning-based approach that is independent of hand-crafted features. However, although such methods of NER employ additional conditional random fields (CRF) to capture important correlations between neighboring labels, they often do not incorporate all the contextual information from text into the deep learning layers.ResultsWe propose herein an NER system for biomedical entities by incorporating n-grams with bi-directional long short-term memory (BiLSTM) and CRF; this system is referred to as a contextual long short-term memory networks with CRF (CLSTM). We assess the CLSTM model on three corpora: the disease corpus of the National Center for Biotechnology Information (NCBI), the BioCreative II Gene Mention corpus (GM), and the BioCreative V Chemical Disease Relation corpus (CDR). Our framework was compared with several deep learning approaches, such as BiLSTM, BiLSTM with CRF, GRAM-CNN, and BERT. On the NCBI corpus, our model recorded an F-score of 85.68% for the NER of diseases, showing an improvement of 1.50% over previous methods. Moreover, although BERT used transfer learning by incorporating more than 2.5 billion words, our system showed similar performance with BERT with an F-scores of 81.44% for gene NER on the GM corpus and a outperformed F-score of 86.44% for the NER of chemicals and diseases on the CDR corpus. We conclude that our method significantly improves performance on biomedical NER tasks.ConclusionThe proposed approach is robust in recognizing biological entities in text.

The rapid advancement of artificial intelligence and intelligent algorithms has led to the widespread adoption of natural language processing technology. Named entity recognition (NER), an important task in this field, takes a pivotal part in information extraction, machine translation, and knowledge graph construction. However, for a small language such as Lao, traditional methods have obvious deficiencies in recognition effectiveness due to its complex syllable structure, fuzzy word boundaries and scarce corpus resources. To address this problem, the study designs a named entity recognition model for Lao that incorporates a Bi-directional Long Short-Term Memory (BiLSTM) network, an attention mechanism and a conditional random field. The model first vectorizes multi-granularity features such as characters, syllables and lexemes through an embedding layer, then captures contextual dependencies by a BiLSTM network, introduces an Attention to highlight key information, and finally establishes a global label transfer relation through a conditional random field to ensure the consistency of the predicted sequence. The findings denote that the model achieves 91.5%, 91.0%, 90.7%, and 90.9% in accuracy rate, precision rate, recall rate, and F1 score, respectively, and the mean absolute error and mean square error are reduced to 0.062 and 0.011, which are significantly better than the comparison model. The results show that combining multi-level feature modeling and global constraint mechanism can effectively enhance the performance of named entity recognition in Lao, which provides a valuable reference and practical path for natural language processing in small languages.

The resident admit notes (RANs) in electronic medical records (EMRs) is first-hand information to study the patient’s condition. Medical entity extraction of RANs is an important task to get disease information for medical decision-making. For Chinese electronic medical records, each medical entity contains not only word information but also rich character information. Effective combination of words and characters is very important for medical entity extraction. We propose a medical entity recognition model based on a character and word attention-enhanced (CWAE) neural network for Chinese RANs. In our model, word embeddings and character-based embeddings are obtained through character-enhanced word embedding (CWE) model and Convolutional Neural Network (CNN) model. Then attention mechanism combines the character-based embeddings and word embeddings together, which significantly improves the expression ability of words. The new word embeddings obtained by the attention mechanism are taken as the input to bidirectional long short-term memory (BI-LSTM) and conditional random field (CRF) to extract entities. We extracted nine types of key medical entities from Chinese RANs and evaluated our model. The proposed method was compared with two traditional machine learning methods CRF, support vector machine (SVM), and the related deep learning models. The result shows that our model has better performance, and the result of our model reaches 94.44% in the F1-score.

In the era of information explosion, named entity recognition (NER) has attracted widespread attention in the field of natural language processing, as it is fundamental to information extraction. Recently, methods of NER based on representation learning, e.g., character embedding and word embedding, have demonstrated promising recognition results. However, existing models only consider partial features derived from words or characters while failing to integrate semantic and syntactic information, e.g., capitalization, inter-word relations, keywords, and lexical phrases, from multilevel perspectives. Intuitively, multilevel features can be helpful when recognizing named entities from complex sentences. In this study, we propose a novel attentive multilevel feature fusion (AMFF) model for NER, which captures the multilevel features in the current context from various perspectives. It consists of four components to, respectively, capture the local character-level (CL), global character-level (CG), local word-level (WL), and global word-level (WG) features in the current context. In addition, we further define document-level features crafted from other sentences to enhance the representation learning of the current context. To this end, we introduce a novel context-aware attentive multilevel feature fusion (CAMFF) model based on AMFF, to fully leverage document-level features from all the previous inputs. The obtained multilevel features are then fused and fed into a bidirectional long short-term memory (BiLSTM)-conditional random field (CRF) network for the final sequence labeling. Extensive experiments on four benchmark datasets demonstrate that our proposed AMFF and CAMFF models outperform a set of state-of-the-art baseline methods and the features learned from multiple levels are complementary.

PurposeSafety management plays an important part in coal mine construction. Due to complex data, the implementation of the construction safety knowledge scattered in standards poses a challenge. This paper aims to develop a knowledge extraction model to automatically and efficiently extract domain knowledge from unstructured texts.Design/methodology/approachBidirectional encoder representations from transformers (BERT)-bidirectional long short-term memory (BiLSTM)-conditional random field (CRF) method based on a pre-training language model was applied to carry out knowledge entity recognition in the field of coal mine construction safety in this paper. Firstly, 80 safety standards for coal mine construction were collected, sorted out and marked as a descriptive corpus. Then, the BERT pre-training language model was used to obtain dynamic word vectors. Finally, the BiLSTM-CRF model concluded the entity’s optimal tag sequence.FindingsAccordingly, 11,933 entities and 2,051 relationships in the standard specifications texts of this paper were identified and a language model suitable for coal mine construction safety management was proposed. The experiments showed that F1 values were all above 60% in nine types of entities such as security management. F1 value of this model was more than 60% for entity extraction. The model identified and extracted entities more accurately than conventional methods.Originality/valueThis work completed the domain knowledge query and built a Q&A platform via entities and relationships identified by the standard specifications suitable for coal mines. This paper proposed a systematic framework for texts in coal mine construction safety to improve efficiency and accuracy of domain-specific entity extraction. In addition, the pretraining language model was also introduced into the coal mine construction safety to realize dynamic entity recognition, which provides technical support and theoretical reference for the optimization of safety management platforms.

In this paper, the named entity recognition (NER) technology is applied to the field of Qu Tan temple murals to identify religious entities in the text. In order to obtain a machine-readable corpus, the relevant books are scanned and optical character recognition is performed, and then the data is manually labeled to form the named entity recognition dataset needed in this study. In this paper, the bidirectional encoder representations from transformers (BERT) model is used to encode the input data, and the obtained word vector can effectively extract the features of the text by making use of the relationship between words. In order to obtain more accurate context information, the bidirectional long short term memory (BiLSTM) is used to extract features and output the prediction label corresponding to each word. After the BiLSTM model, the conditional random field (CRF) algorithm is joined, which will correct the score of the output, and the output is the optimal label sequence. In this study, the performance of the model is also improved by data enhancement.

Chinese pesticide named-entity recognition (NER) aims to identify named entities related to pesticide properties from unstructured Chinese pesticide information texts. In view of the characteristics of massive, fragmented, professional, and complex semantic relationships of pesticide information data, a deep learning method based on multi-feature fusion was applied to improve the accuracy of pesticide NER. In this study, the pesticide data set is manually annotated by the begin inside outside (BIO) sequence annotation scheme. Bi-directional long short-term memory (BiLSTM) and iterated dilated convolutional neural networks (IDCNN) combined with conditional random field (CRF) form the model BiLSTM-IDCNN-CRF, and it is applied to implement named-entity recognition in Chinese pesticide data sets. IDCNN is introduced to enhance the semantic representation ability and local feature capture ability of the text. BiLSTM network and IDCNN network are combined to obtain the long-distance dependence relationship and context features of different granularity of pesticide data text. Finally, CRF is used to implement the sequence labeling task. According to the experiment results, the accuracy rate, recall rate, and F1 score of the BiLSTM-IDCNN-CRF model in the Chinese pesticide data set were 78.59%, 68.71%, and 73.32%, respectively, which are significantly better than other compared models. Experiments show that the BiLSTM-IDCNN-CRF model can effectively identify and extract entities from Chinese pesticide information text data, which is helpful in constructing the pesticide information knowledge graph and intelligent question-answering.

Automatic Parts of Speech(POS) tagging is one of the most fundamental tasks for a language in Natural Language Processing(NLP), which acts as a feature for solving advanced NLP tasks. Named Entity Recognition(NER) is another essential task of NLP for information retrieval. Researchers could not find up to the mark solution yet on these two tasks for Bangla language compared to other languages, for instance, English, Ger-man. Moreover, many solutions heavily depend on handcrafted features that require strong linguistic expertise. As these two sequence labeling tasks are similar, In this work, two different datasets of POS tagging and NER were prepared, and different deep neural network approaches studied for solving these two tasks separately. All of the approaches were end to end and did not need any handcrafted feature like word suffixes or affixes, gazetteers, dictionary. This study came up with an end to end solution using deep neural network-based model consisting of Bi-directional Long short-term memory(BLSTM), Convolutional Neural Network(CNN) and Conditional Random Field(CRF). The proposed model trained on respected datasets achieved an accuracy of 93.86% on POS tagging and a strict f1 score of 0.6285 on NER on prepared datasets, respectively.

<p>The traditional Chinese Named Entity Recognition (NER) method is difficult to define the entity category of the word according to the specific language environment, and the category is ambiguous, so it is difficult to accurately identify the entity. Named entity recognition based on deep learning can find entity categories in text, so it has received widespread attention. On the basis of the neural network model, this paper proposes a model based on TextCNN-BiLSTM-CRF and text classification (TextCNN-BiLSTM-TC-CRF) for Chinese NER. First, the TextCNN model is used to extract the word vector information of the text data; secondly, bidirectional LSTM is used the model extracts the contextual features of the text; then the neural network model is used to automatically extract the word features and the global features of the text for text classification; finally, the text sequence labeling and entity recognition are performed. Experiments verify that on a large-scale Chinese NER data set, the entity recognition model proposed in this paper has better evaluation indicators than other algorithms, with an F1 score of 98.7%.</p> <p> </p>

Information extraction (IE) is a textual information processing task concerned with the automatic extraction of entity mentions and relational structures from documents, which has been widely studied and applied in various fields such as biomedicine and news. In this paper, we extend this technique to biographies and propose a new BioIE3 dataset to extract the authors' experiences (including birth, education, work and awards) from their profiles in the scientific literatures. Named entity recognition (NER) and relation extraction (RE) tasks are performed simultaneously in a neural architecture based on the Skip-Gram Word2vec, Bidirectional Long Short Term Memory (BiLSTM) and Conditional Random Fields (CRF) models. On the basis of the above work, negative examples are especially introduced into the training stage to effectively improve and increase the robustness of the model.

Character level and word level embedding with bidirectional LSTM – Dynamic recurrent neural network for biomedical named entity recognition from literature

BackgroundThe unstructured data of Chinese cancer electronic health records (EHRs) contains valuable medical expertise. Accurate medical entity recognition is crucial for building a medical-assisted decision system. Named entity recognition (NER) in cancer EHRs typically uses general models designed for English medical records. There is a lack of specialized handling for cancer-specific records and limited application to Chinese medical records.ObjectiveThis study aims to propose a specific NER model to enhance the recognition of medical entities in Chinese cancer EHRs.MethodsDesensitized inpatient EHRs related to breast cancer were collected from a leading hospital in Beijing. Building upon the MC Bidirectional Encoder Representations from Transformers (BERT) foundation, the study further incorporated a Chinese cancer corpus for pretraining, resulting in the construction of the ChCancerBERT pretrained model. In conjunction with dilated-gated convolutional neural networks, bidirectional long short-term memory, multihead attention mechanism, and a conditional random field, this model forms a multimodel, multilevel integrated NER approach.ResultsThis approach effectively extracts medical entity features related to symptoms, signs, tests, treatments, and time in Chinese breast cancer EHRs. The entity recognition performance of the proposed model surpasses that of the baseline model and other models compared in the experiment. The F1-score reached 86.93%, precision reached 87.24%, and recall reached 86.61%. The model introduced in this study demonstrates exceptional performance on the CCKS2019 dataset, attaining a precision rate of 87.26%, a recall rate of 87.27%, and an impressive F1-score of 87.26%, surpassing that of existing models.ConclusionsThe experiments demonstrate that the approach proposed in this study exhibits excellent performance in NER within breast cancer EHRs. This advancement will further contribute to clinical decision support for cancer treatment and research. In addition, the study reveals that incorporating domain-specific corpora in clinical NER tasks can further enhance the performance of BERT models in specialized domains.

To solve the problem that it is difficult to identify the key Chinese entity information in the hazard and operability analysis text, a deep neural network model based on bidirectional long short-term memory and conditional random field (BiLSTM-CRF) is proposed to identify key named entities in the text. In the word vector pre-training process, bidirectional encoder representation from transformers (BERT) model is used to pre-train word vectors instead of the static word vectors in the traditional word2vec model, and then obtain context-related dynamic word vectors, to improve the representational ability of word vectors, and solve the problem of word boundary division when word vectors are used in Chinese corpus training. This model has the ability of complete the task of Chinese named entity recognition. The F1 value on the test corpus reaches 93.31%, which is 21.82% higher than conditional random field (CRF) Baseline, and 4.49% higher than the traditional BiLSTM-CRF model. The experimental results show that the BERT-BiLSTM-CRF model is effective for the named entity recognition (NER) task of the hazard and operability analysis text, and it is helpful to automatically extract the relationship between the entities in the hazard and operability analysis text and build safety analysis knowledge graph.

With the increasing number of network security threats and the frequent occurrence of software vulnerability attacks, the effective management and large-scale retrieval of vulnerability data have become urgent needs. Existing vulnerability information is scattered across heterogeneous sources and is difficult to integrate, which in turn makes it hard for security analysts to quickly retrieve and analyze relevant security knowledge. To address this problem, this paper proposes a method to construct a vulnerability knowledge graph by integrating multi-source vulnerability data, combining graph embedding technology with large language model reasoning to aggregate, infer, and enrich vulnerability knowledge. Experiments demonstrated that our domain-tuned Bidirectional Long Short-Term Memory–Conditional Random Field (BiLSTM-CRF) named entity recognition (NER), enhanced with a cybersecurity dictionary, achieved a 90.1% F1-score for entity extraction. For link prediction, a hybrid Graph Attention Network fused with GPT-3 reasoning boosted Hits1 by 0.137, Hits3 by 0.116, and Hits10 by 0.101 over the baseline. These results confirm that our approach markedly enhanced entity identification and relationship inference, yielding a more complete and dynamically updatable cybersecurity knowledge graph.

With Entity recognition is an important basic tool for many natural language processing tasks such as information extraction, question answering systems, and syntactic analysis. Entity recognition is divided into general field and specific field. There were different recognition methods for different fields. At present, one of the more common entity recognition technologies is a rule-based method, and the other is a statistical method. Due to data sets scarcity in the aerospace engine field, there is a lack of corresponding entity recognition research in the aerospace engine field. This paper proposes a statistical-based entity recognition method to identify entities in a specific field based on conditional random fields. This method constructs a training set in the aerospace engine field through manual annotation, extracts entities in the aerospace engine field, uses words, word frequency, and part-of-speech features respectively, and conducts experiments. This method can more accurately identify entities in the aerospace engine field, and finally make the accuracy rate, Recall rate and F value reach 92.7%, 87.1%, 89.8%.