Employing Auto-annotated Data for Person Name Recognition in Judgment Documents

In this paper, we focus on the task of person name recognition (PNR) in Chinese text, which is an important part of Named entity recognition (NER). Chinese word segmentation makes the sequence tagging of PNR more accurate than character-based PNR, but it also brings more word-level ambiguities. For reducing the negative effect of word segmentation for PNR, we reinforce the analysis of ambiguities and propose a model to deal with them. Experimental results on People's Daily corpus show that the proposed model is effective in PNR.

In recent years, distantly-supervised relation extraction has achieved a certain success by using deep neural networks. Distant Supervision (DS) can automatically generate large-scale annotated data by aligning entity pairs from Knowledge Bases (KB) to sentences. However, these DS-generated datasets inevitably have wrong labels that result in incorrect evaluation scores during testing, which may mislead the researchers. To solve this problem, we build a new dataset NYTH, where we use the DS-generated data as training data and hire annotators to label test data. Compared with the previous datasets, NYT-H has a much larger test set and then we can perform more accurate and consistent evaluation. Finally, we present the experimental results of several widely used systems on NYT-H. The experimental results show that the ranking lists of the comparison systems on the DS-labelled test data and human-annotated test data are different. This indicates that our human-annotated data is necessary for evaluation of distantly-supervised relation extraction.

Named Entity Recognition (NER) is one of the first stages in deep language understanding yet current NER models heavily rely on human-annotated data. In this work, to alleviate the dependence on labeled data, we propose a Local Additivity based Data Augmentation (LADA) method for semi-supervised NER, in which we create virtual samples by interpolating sequences close to each other. Our approach has two variations: Intra-LADA and Inter-LADA, where Intra-LADA performs interpolations among tokens within one sentence, and Inter-LADA samples different sentences to interpolate. Through linear additions between sampled training data, LADA creates an infinite amount of labeled data and improves both entity and context learning. We further extend LADA to the semi-supervised setting by designing a novel consistency loss for unlabeled data. Experiments conducted on two NER benchmarks demonstrate the effectiveness of our methods over several strong baselines. We have publicly released our code at https://github.com/GT-SALT/LADA

Most Graph Neural Networks (GNNs) follow the message-passing scheme. Residual connection is an effective strategy to tackle GNNs' over-smoothing issue and performance reduction issue on non-homophilic networks. Unfortunately, the coarse-grained residual connection still suffers from class-imbalanced over-smoothing issue, due to the fixed and linear combination of topology and attribute in node representation learning. To make the combination flexible to capture complicated relationship, this paper reveals that the residual connection needs to be node-dependent, layer-dependent, and related to both topology and attribute. To alleviate the difficulty in specifying complicated relationship, this paper presents a novel perspective on GNNs, i.e., the representations of one node in different layers can be seen as a sequence of states. From this perspective, existing residual connections are not flexible enough for sequence modeling. Therefore, a novel node-dependent residual connection, i.e., Long Short-Term Graph Memory Network (LSTGM) is proposed to employ Long Short-Term Memory (LSTM), to model the sequence of node representation. To make the graph topology fully employed, LSTGM innovatively enhances the updated memory and three gates with graph topology. A speedup version is also proposed for effective training. Experimental evaluations on real-world datasets demonstrate their effectiveness in preventing over-smoothing issue and handling networks with heterophily.

The research study is to predict human activity recognition and analyze their strength in machine learning (ML) processes to achieve precision and also to determine performance of human activities. The framework to classify the Novel Long Short Term Memory (LSTM) and Decision Tree (DT) to perform all measures. This research study used the Novel Long Short Term Memory and Decision Tree to perform the required actions to accurately identify body movements in adjacent film sequence. Recent study findings and a threshold of 0.05%, as well as 95% confidence interval and standard parameter estimates, were castoff in a study 47 samples divided into two groups calculated using the G-power of 80%. Since the novel Long Short Term Memory (LSTM) algorithm and local invariant methods have achieved an accuracy of 93.83% in predicting the activity analysis, this research needs to obtain improved accuracy to perform activity prediction by using the Decision Tree algorithm. When applied to human activity analysis, the Novel Long Short Term Memory technique was shown to have an accuracy of 90.12 percent, with a 95% confidence interval including the value 0.045 (p 0.05). Researchers found that when comparing two algorithms for analyzing human behaviour, Long Short Term Memory (LSTM) performed much better than Decision Tree (DT).

Our study focuses on Traditional Chinese Medical (TCM) named entity recognition (NER), which involves identifying and extracting specific entity names from TCM record. This task has significant implications for doctors and researchers, as it enables the automated identification of relevant TCM terms, ultimately enhancing research efficiency and accuracy. However, the current Bidirectional Encoder Representations from Transformers-Long Short Term Memory-Conditional Random Fields (BERT-LSTM-CRF) model for TCM NER is constrained by a traditional structure, limiting its capacity to fully harness the advantages provided by Bidirectional Encoder Representations from Transformers (BERT) and long short term memory (LSTM) models. Through comparative experiments, we also observed that the straightforward superimposition of models actually leads to a decrease in recognition results. To optimize the structure of the traditional BERT-BiLSTM-CRF model and obtain more effective text representations, we propose the Dyn-Att Net model, which introduces dynamic attention and a parallel structure. By integrating BERT and LSTM models with the dynamic attention mechanism, our model effectively captures semantic, contextual, and sequential relations within text sequences, resulting in high accuracy. To validate the effectiveness of our model, we compared it with nine other models in TCM dataset namely the publicly available PaddlePaddle dataset. Our Dyn-Att Net model, based on BERT, outperforms the other models, achieving an F1 score of 81.91%, accuracy of 92.06%, precision of 80.26%, and recall of 83.76%. Furthermore, its robust generalization capability is substantiated through validation on the APTNER, MSRA, and EduNER datasets. Overall, the Dyn-Att Net model not only enhances NER accuracy within the realm of traditional Chinese medicine, but also showcases considerable potential for cross-domain generalization. Moreover, the Dyn-Att Net model's parallel architecture facilitates efficient computation, contributing to time-saving efforts in NER tasks.

Named Entity Recognition (NER) on Clinical Electronic Medical Records (CEMR) is a fundamental step in extracting disease knowledge by identifying specific entity terms such as diseases, symptoms, etc. However, the state-of-the-art NER methods based on Long Short-Term Memory (LSTM) fail to fully exploit GPU parallelism under the massive medical records. Although a novel NER method based on Iterated Dilated CNNs (ID-CNNs) can accelerate network computing, it tends to ignore the word-order feature and semantic information of the current word. In order to enhance the performance of ID-CNNs-based models on NER tasks, an attention-based ID-CNNs-CRF model which combines word-order feature and local context is proposed. Firstly, Position Embedding is utilized to fuse word-order information. Secondly, ID-CNNs architecture is used to rapidly extract global semantic information. Simultaneously, the attention mechanism is employed to pay attention to the local context. Finally, we apply the CRF to obtain the optimal tag sequence. Experiments conducted on two CEMR datasets show that our model outperforms traditional ones. The F1-scores of 94.55% and 91.17% are obtained respectively on these two datasets and both are better than LSTMs-based models.

Abstract: This research is driven by the challenges faced by Universitas Lancang Kuning (UNILAK) in attracting applicants amidst intense competition, especially after the government's policy opened independent pathways to State Universities (PTN) from 2022-2023, which impacted private university applicant numbers. To address this and support strategic planning, this study aims to predict the trend of prospective students applying to all study programs at UNILAK for the period 2025-2027. Two time series models were employed: ARIMA (AutoRegressive Integrated Moving Average) and LSTM (Long Short-Term Memory). Applicant data from 2019 to 2024 was used to build the model. The Augmented Dickey-Fuller (ADF) test confirmed the data's stationarity with a p-value of 0.0. ACF and PACF analyses determined the ARIMA parameters as p=1, d=1, q=1. The LSTM model was trained to capture more complex data patterns. ARIMA predictions for 2025, 2026, and 2027 are 3298.66, 3362.33, and 3371.30, respectively. LSTM predictions for the same years are 3335.64, 3476.52, and 3518.42. Evaluation using Root Mean Squared Error (RMSE) showed ARIMA (RMSE=588.72) to be more accurate than LSTM (RMSE=653.96). Nevertheless, LSTM provided a more optimistic prediction. This study concludes that ARIMA is better suited for short-term planning, while LSTM can be used for more ambitious long-term strategies. Keywords: arima; LSTM; applicants; prediction; university Abstrak: Penelitian ini didorong oleh tantangan Universitas Lancang Kuning (UNILAK) dalam menarik pendaftar di tengah persaingan ketat, khususnya setelah kebijakan pemerintah membuka jalur mandiri ke Perguruan Tinggi Negeri (PTN) sejak 2022-2023, yang menyebabkan penurunan jumlah pendaftar di universitas swasta. Untuk mendukung perencanaan strategis, studi ini bertujuan memprediksi tren jumlah calon mahasiswa yang mendaftar ke seluruh program studi di UNILAK untuk periode 2025-2027.Dua model deret waktu digunakan: ARIMA (AutoRegressive Integrated Moving Average) dan LSTM (Long Short-Term Memory). Data jumlah pendaftar dari 2019 hingga 2024 digunakan untuk membangun model. Uji Augmented Dickey-Fuller (ADF) menunjukkan data stasioner dengan p-value 0,0. Analisis ACF dan PACF menentukan parameter ARIMA sebagai p=1, d=1, q=1. Model LSTM dilatih untuk menangkap pola data yang lebih kompleks.Prediksi ARIMA untuk 2025, 2026, dan 2027 adalah 3298.66, 3362.33, dan 3371.30. Prediksi LSTM untuk tahun yang sama adalah 3335.64, 3476.52, dan 3518.42. Evaluasi menggunakan Root Mean Squared Error (RMSE) menunjukkan ARIMA (RMSE=588.72) lebih akurat daripada LSTM (RMSE=653.96). Meskipun demikian, LSTM memberikan prediksi yang lebih optimis. Studi ini menyimpulkan ARIMA lebih cocok untuk perencanaan jangka pendek, sementara LSTM dapat digunakan untuk strategi jangka panjang yang ambisius. Kata kunci: arima; LSTM; pendaftar; prediksi; universitas

Evaluation of clinical named entity recognition methods for Serbian electronic health records

Named entity recognition (NER) plays a vital role in extracting critical medical entities from health care records, facilitating applications such as clinical decision support and data mining. Developing robust NER models for low-resource languages, such as Estonian, remains a challenge due to the scarcity of annotated data and domain-specific pretrained models. Large language models (LLMs) have proven to be promising in understanding text from any language or domain. This study addresses the development of medical NER models for low-resource languages, specifically Estonian. We propose a novel approach by generating synthetic health care data and using LLMs to annotate them. These synthetic data are then used to train a high-performing NER model, which is applied to real-world medical texts, preserving patient data privacy. Our approach to overcoming the shortage of annotated Estonian health care texts involves a three-step pipeline: (1) synthetic health care data are generated using a locally trained GPT-2 model on Estonian medical records, (2) the synthetic data are annotated with LLMs, specifically GPT-3.5-Turbo and GPT-4, and (3) the annotated synthetic data are then used to fine-tune an NER model, which is later tested on real-world medical data. This paper compares the performance of different prompts; assesses the impact of GPT-3.5-Turbo, GPT-4, and a local LLM; and explores the relationship between the amount of annotated synthetic data and model performance. The proposed methodology demonstrates significant potential in extracting named entities from real-world medical texts. Our top-performing setup achieved an F1-score of 0.69 for drug extraction and 0.38 for procedure extraction. These results indicate a strong performance in recognizing certain entity types while highlighting the complexity of extracting procedures. This paper demonstrates a successful approach to leveraging LLMs for training NER models using synthetic data, effectively preserving patient privacy. By avoiding reliance on human-annotated data, our method shows promise in developing models for low-resource languages, such as Estonian. Future work will focus on refining the synthetic data generation and expanding the method's applicability to other domains and languages.

Online leakage current classification using convolutional neural network long short-term memory for high voltage insulators on web-based service

Recognition of chinese personal names becomes a difficult and key point in chinese unknown word recognition. This paper explored the context boundary of names and the law of names. The context boundary of names is concentrated, which can reduce recognition errors brought up by Forward Maximum Matching Segmentation; from real text corpus, we discover that names begin with surname, dislocation characters of names reach 70.83%, and rare characters of names reach 9.42%. This paper improved the Forward Maximum Matching Segmentation and implemented a name recognition test based on CRFs, which was combined with the surname, the context boundary, the dislocation character and the rare character. The open test shows that recall reaches 91.24% from BakeOff-2005.KeywordsChinese personal namesSegmentationCRFsDislocation characterRare character

Long short-term memory - Fully connected (LSTM-FC) neural network for PM2.5 concentration prediction

News text is text that contains information about an incident that is submitted in writing or not. In the news text there is a lot of unstructured information and makes it difficult for people to know what entities are contained in the news text. To overcome these problems, information extraction is carried out which is useful for adapting information entities in the form of structured text. In this study, information extraction was carried out using the Long Short-Term Memory (LSTM) method in a news data related to Telkom University. Therefore, in this study, the development of an information extraction system was carried out by focusing on the Person entity using the LSTM method. The test results obtained an F1 Score of 81% with a tuning parameter 256 word embedding size, 32 batch, 150 epochs, 0.1 dropout, 150 units, and 1 layer on LSTM.

Time series data has attached extensive attention as multi-domain data, but it is difficult to analyze due to its high dimension and few labels. Self-supervised representation learning provides an effective way for processing such data. Considering the frequency domain features of the time series data itself and the contextual feature in the classification task, this paper proposes an unsupervised Long Short-Term Memory (LSTM) and contrastive transformer-based time series representation model using contrastive learning. Firstly, transforming data with frequency domain-based augmentation increases the ability to represent features in the frequency domain. Secondly, the encoder module with three layers of LSTM and convolution maps the augmented data to the latent space and calculates the temporal loss with a contrastive transformer module and contextual loss. Finally, after self-supervised training, the representation vector of the original data can be got from the pre-trained encoder. Our model achieves satisfied performances on Human Activity Recognition (HAR) and sleepEDF real-life datasets.