High-Quality Article Classification Based on Named Entities of Knowledge Graph and Multi-head Attention

This study proposes a deep learning method based on ResNet, Transformer, and Multi-Head Attention for food quality inspection. With the growing severity of food safety issues, traditional food quality inspection methods have significant shortcomings in efficiency and accuracy. Therefore, there is an urgent need to develop automated, efficient, and accurate food quality inspection technologies. Our method combines a deep convolutional neural network (ResNet), a self-attention mechanism (Transformer), and a multi-head attention mechanism (Multi-Head Attention). It effectively extracts detailed features from food images while enhancing attention to global image information and important regions, effectively identifying mold, contamination, and spoilage in food. Experimental results demonstrate that this method outperforms traditional deep learning models in metrics such as precision, recall, and F1 score. Ablation experiments also validate the contribution of each module to the performance improvement. This model provides a more accurate and automated solution for food quality inspection and has broad application prospects.

In this paper, we introduce text sentiment analysis model GAT-SA (Graph Attention Network for Sentiment Analysis) to better recognize subtle emotional expressions. The conventional way for sentiment analysis lacks of deep semantic relationship between words, which is the key point to analyzing some fine-grained emotions. Therefore, we first make word text to a form of graph, the node presents words and the edge presents syntactic dependencies. With graph attention mechanism, GAT dynamically adjust the weight between neighbors so that it can better capture sentiment. With the multi-head attention mechanism, GAT-SA can better capture long range dependency and context. Experimental results indicate that GAT-SA performs better on sentiment analysis dataset IMDB, Twitter, with higher accuracy and F1 scores compared with traditional methods.

To better understand and evaluate the GNSS positioning performance, it is convenient to adopt corresponding measures to reduce the impact of errors on positioning. A GNSS positioning error estimation scheme based on an improved multi-layer perceptron model is proposed. The multi-head attention mechanism and gating operation are integrated into the multi-layer perceptron model to adaptively select and filter features, enhancing the model’s ability to understand input features. First, the original positioning error of the satellite is obtained through the Kalman filter positioning method. The data are then preprocessed to extract available features. Finally, the features are input into the constructed model for training and testing to obtain the estimated positioning error value. Two types of comparative experiments were completed. The performance of the presented model is evaluated by the root mean square error. Experimental results show that the proposed method performs well in terms of performance indicators, and has obvious advantages over other state-of-the-art methods. In particular, the root mean square error of the presented method in the first dataset is 0.239 m, which is 39.2% and 17% lower than the current state-of-the-art long short-term memory network and convolutional neural network, respectively. The presented method can provide higher-precision estimated values for studying the GNSS positioning error estimation problem.

To overcome the limitations of existing deep learning-based image recognition approaches for diagnosing the operating conditions of pumping wells–namely, slow processing speed and high parameter complexity–this paper proposes a lightweight diagnostic method based on a one-dimensional convolutional neural network (1D CNN). Unlike traditional image-based techniques, the proposed method eliminates the need to convert raw data into dynamometer card images, thereby reducing computational overhead and significantly decreasing the number of model parameters. The architecture integrates 1D CNN with a bidirectional long short-term memory (BiLSTM) network to effectively capture both local patterns and global temporal dependencies inherent in different operating states of pumping wells. Additionally, a multi-head attention (MHA) mechanism is incorporated to enhance the representation of critical time points, thereby improving the model's sensitivity to key features and its overall classification performance. Experiments conducted on an augmented dataset–generated via a simulation-based algorithm–demonstrate that the proposed 1DCNN-BiLSTM-MHA model achieves a classification accuracy of 99.22%. Experimental results demonstrate that the proposed model possesses strong feature learning capabilities. Comparative analysis with several baseline models highlights its superior diagnostic performance and reliability, along with reduced parameter count and improved computational speed.

Knowledge reasoning technology is a method proposed to solve missing problems in knowledge graphs, which has been developing in recent years. In order to solve the problems of low accuracy, poor interpretability, and weak applicability in knowledge reasoning, a knowledge reasoning model called MLALogic is proposed, which combines HousE with multi-head latent attention mechanism. It consists of a rule generator with multi-head latent attention mechanism and a rule predictor with HousE embedding. The rule generator generates the rules required for reasoning and passes them into the predictor, which updates and then obtains scores for different rules, After that, the generator and predictor are continuously trained and optimised by the EM algorithm. Specifically, The model employs a multi-head latent attention mechanism to select more relevant relationships as rules, thereby improving model accuracy and reducing training costs. Additionally, the integration of HousE enhances flexibility in handling complex and multi-relational data.

Major Depressive Disorder (MDD) significantly impacts global health, impairing individual functioning and increasing socioeconomic burden. Developing innovative, interpretable approaches for its identification is essential for improving diagnosis and guiding treatment. This study introduces a novel framework designed to classify MDD using resting-state functional MRI (rs-fMRI) data. Our framework follows three stages: First, Node2Vec extracts rich, low-dimensional brain region embeddings from functional connectivity (FC) networks, capturing their complex topological information. Second, these informative embeddings feed a Graph Attention Network (GAT) which, via multi-head attention, identifies and weighs discriminative inter-regional functional connections, refining them into a potent graph representation. Third, these GAT-derived representations are classified by an ensemble classifier (Random Forest, SVM, MLP) for robust MDD identification. The model achieved classification accuracies of 78.73% and 92.94% on the REST-meta-MDD and SRPBS-MDD datasets, respectively. Moreover, the attention mechanism revealed that resting-state functional connectivity of regions within the Default Mode Network (DMN) and Frontoparietal Network (FPN) were among the most discriminative features for distinguishing MDD from healthy controls. The attention mechanism enhanced interpretability by highlighting significant brain regions linked to MDD. Compared to traditional methods, this GNN-based approach effectively captures complex brain connectivity patterns and offers improved interpretability, ultimately aiding healthcare professionals in diagnosing MDD more accurately.

Piwi interacting RNA (piRNA) is a type of small non coding RNA with a length of 24-32 nucleotides, mainly expressed in germ cells. Its abnormal expression is closely related to various diseases such as cancer and neurodegenerative diseases. Although biological experiments are the gold standard for identifying the association between piRNA and disease (PDA), their high cost and long cycle limit research progress. Therefore, computational models have become an important tool for assisting in predicting PDA. However, existing computational methods generally suffer from issues such as insufficient feature extraction and imbalanced data. This article proposes a prediction model based on the fusion of graph convolutional network and attention mechanism - RandGCN. This model combines piRNA sequence embedding, heterogeneous graph construction based on random walks, multi-layer graph convolution feature extraction, and multi head attention mechanism with gating units, effectively improving the accuracy and robustness of PDA prediction. The experimental results on the MNDR dataset show that RandGCN performs well in both AUC and AUPR values, demonstrating excellent predictive performance and potential applications.

Considering precision agriculture, recent technological developments have sparked the emergence of several new tools that can help to automate the agricultural process. For instance, accurately detecting and counting apples in orchards is essential for maximizing harvests and ensuring effective resource management. However, there are several intrinsic difficulties with traditional techniques for identifying and counting apples in orchards. To identify, recognize, and detect apples, apple target detection algorithms, such as YOLOv7, have shown a great deal of reflection and accuracy. But occlusions, electrical wiring, branches, and overlapping pose severe issues for precisely detecting apples. Thus, to overcome these issues and accurately recognize apples and find the depth of apples from drone-based videos in complicated backdrops, our proposed model combines a multi-head attention system with the YOLOv7 object identification framework. Furthermore, we provide the ByteTrack method for apple counting in real time, which guarantees effective monitoring of apples. To verify the efficacy of our suggested model, a thorough comparison assessment is performed with several current apple detection and counting techniques. The outcomes adequately proved the effectiveness of our strategy, which continuously surpassed competing methods to achieve exceptional accuracies of 0.92, 0.96, and 0.95 with respect to precision, recall, and F1 score, and a low MAPE of 0.027, respectively.

BackgroundWith the rapid expansion of biomedical literature, biomedical information extraction has attracted increasing attention from researchers. In particular, relation extraction between 2 entities is a long-term research topic.ObjectiveThis study aimed to perform 2 multiclass relation extraction tasks of Biomedical Natural Language Processing Workshop 2019 Open Shared Tasks: relation extraction of Bacteria-Biotope (BB-rel) task and binary relation extraction of plant seed development (SeeDev-binary) task. In essence, these 2 tasks are aimed at extracting the relation between annotated entity pairs from biomedical texts, which is a challenging problem.MethodsTraditional research methods adopted feature- or kernel-based methods and achieved good performance. For these tasks, we propose a deep learning model based on a combination of several distributed features, such as domain-specific word embedding, part-of-speech embedding, entity-type embedding, distance embedding, and position embedding. The multi-head attention mechanism is used to extract the global semantic features of an entire sentence. Meanwhile, we introduced a dependency-type feature and the shortest dependency path connecting 2 candidate entities in the syntactic dependency graph to enrich the feature representation.ResultsExperiments show that our proposed model has excellent performance in biomedical relation extraction, achieving F1 scores of 65.56% and 38.04% on the test sets of the BB-rel and SeeDev-binary tasks. Especially in the SeeDev-binary task, the F1 score of our model is superior to that of other existing models and achieves state-of-the-art performance.ConclusionsWe demonstrated that the multi-head attention mechanism can learn relevant syntactic and semantic features in different representation subspaces and different positions to extract comprehensive feature representation. Moreover, syntactic dependency features can improve the performance of the model by learning dependency relation between the entities in biomedical texts.

The development of social media has changed the way the public expresses political opinions, especially regarding the evaluation of President Joko Widodo’s (Jokowi) leadership after his term. Platform X (formerly Twitter) has become the primary source of public opinion data, but the use of informal language and sarcasm makes accurate sentiment analysis challenging. This study creates a sentiment analysis model that uses deep learning with a CNN-BiLSTM structure and a multi-head attention mechanism. The dataset consists of 52,643 tweets that have been labeled and embedded using IndoBERT. To address class imbalance, the SMOTE method was applied to the training data, enabling the model to better learn from minority classes. The results indicate that the model achieves a high accuracy of 98.78%, with an average precision, recall, and F1-score of 0.98. These findings indicate that the model is not only accurate but also reliable in distinguishing each sentiment class. A comparison with other model variants suggests that the complete combination of CNN-BiLSTM and Multi-Head Attention delivers the best performance, although the improvement is relatively small.

How to enable the computer to accurately analyze the emotional information and story background of characters in Qin opera is a problem that needs to be studied. To promote the artistic inheritance and cultural emotion color dissemination of Qin opera, an emotion analysis model of Qin opera based on attention residual network (ResNet) is presented. The neural network is improved and optimized from the perspective of the model, learning rate, network layers, and the network itself, and then multi-head attention is added to the ResNet to increase the recognition ability of the model. The convolutional neural network (CNN) is optimized from the internal depth, and the fitting ability and stability of the model are enhanced through the ResNet model. Combined with the attention mechanism, the expression of each weight information is strengthened. The multi-head attention mechanism is introduced in the model and a multi-head attention ResNet, namely, MHAtt_ResNet, is proposed. The network structure can effectively identify the features of the spectrogram, improve the weight information of spectrogram features, and deepen the relationship between distant information in long-time series. Through experiments, the proposed model has high emotional classification accuracy for Qin opera, and with the increase of the number of data sets, the model will train a better classification effect.

Industrial water withdrawal prediction is the cornerstone of water resources monitoring and early warning, as well as a key task in implementing rigid constraints on water resources. Although many factors may influence water withdrawal, they are difficult to obtain. Extracting the features in the historical water withdrawal data itself is a more convenient alternative way to achieve accurate prediction. However, due to the lack of significant regularity and periodicity in industrial water data, feature extraction is challenging. In this paper, a multi-head attention encoder (MAEN) model consisting of multi-head attention mechanisms and feed-forward neural networks is proposed to enable more convenient and accurate forecasting. The multi-head attention mechanism allows to focus on different parts of the data simultaneously, detecting complex dependencies, and extracting key features from historical data to enable more efficient feature extraction. Applied to water withdrawal data from real factories, the proposed model outperforms common time series prediction models including artificial neural network, long short-term memory, and gate recurrent unit. Compared to the best-performing comparison model, MAEN shows a reduction of mean squared error by 3.2% for 1 day ahead predictions, and 8.9% for 7 days ahead predictions, illustrating its superior performance in industrial water withdrawal prediction.

Obtaining robust feature representations from multi-position wearable sensory data is challenging in human activity recognition (HAR) since data from different positions can have unordered implicit correlations. Graph neural networks (GNNs) represent data as structured graphs by mining complex relationships and interdependency via message passing between the nodes of graphs. This paper proposes a novel framework (MhaGNN) that combines GNNs and the multi-head attention mechanism, aiming to learn more informative representations for multi-position HAR tasks. The MhaGNN framework takes the sensor channels from multiple wearing positions as nodes to construct graph-structured data from the spatial dimension. Besides, the multi-head attention mechanism is introduced to complete the message passing and aggregation of the graphs for spatial-temporal feature extraction. The MhaGNN learns correlations among sensor channels that can be used as compensatory features together with the captured features from each single sensor channel to enhance HAR. Experimental evaluations on three publicly available HAR datasets and a ground-truth dataset demonstrate that our proposed MhaGNN achieves state-of-the-art recognition performance with the captured rich features, including PAMAP2, OPPORTUNITY, MHAEATH and MPWHAR.

<title>Abstract</title> Signals from global navigation satellite systems (GNSSs) in urban areas suffer from serious multipath errors caused by building blockages and reflections. The use of deep neural networks offers great potential for predicting and eliminating complex multipath/non-line-of-sight (NLOS) errors. However, existing methods for predicting the original signals face two remaining challenges. The first challenge is an inability to effectively exploit irregular GNSS measurement data caused by an inconsistent number of visible satellites in different epochs. The second challenge is degradation in the generalization performance of the multipath/NLOS prediction model when using data collected from different locations and periods. To address these challenges, this paper proposes a novel graph transformer neural network (GTNN) for predicting satellite visibility that effectively learns environment representations from irregular GNSS measurements to both alleviate multipath interference and improve the generalization performance of the multipath prediction model. To learn from irregular GNSS measurements, a sky satellite graph is constructed as input to a graph neural network by using satellites captured in the same epoch, which can represent the spatial relationships between satellites and enable the model to learn satellite-related features sufficiently well. To improve the generalization ability of our multipath prediction model, a multihead attention mechanism is introduced to aggregate satellite node information by computing the correlation between satellites to extract the environment representation around the receiver. Based on the constructed sky satellite graph and the multihead attention mechanism, our novel GTNN for predicting satellite visibility can not only handle irregular GNSS measurements but can also learn an environment representation via graph attention. Comparative experiments were conducted on real-world GNSS measurement data in urban areas, demonstrating that the proposed method can achieve an accuracy exceeding 96% for satellite visibility prediction and obtain better generalization performance than existing multipath prediction methods. Moreover, the attention weights among satellites were visualized to demonstrate the environment representation learned by the GTNN from the sky satellite graph.

As one of the core tasks of natural language processing technology, text classification methods general-ly face the problems of insufficient global semantic capture and limited feature focusing ability when processing long texts or complex semantics. To address this issue, a deep learning model that integrates improved convolutional neural networks, unbalanced bidirectional long short-term memory networks, and multi-head attention mechanisms is proposed. Utilizing an improved bidirectional long short-term memory network to capture global semantic information, while dynamically focusing on key features through a multi head attention mechanism to enhance the model's adaptability to classification tasks. The performance of the model is validated through experiments on AG News (short text) and IMDb (long text) datasets. The results show that in short text classification, the proposed method has an accu-racy rate of 96% and a classification error rate of only 1.46%. In the task of long text classification, the method proposed in the study has a product under the curve of 0.98. In adversarial attack testing, the accuracy rates of adversarial samples generated by different methods are 92.85% and 90.63%, respec-tively, with the lowest robustness degradation rates of 3.72% and 5.49%, respectively. In cross domain generalization testing, it shows the least classification errors and superior cross domain adaptability. These results validate the high performance, robustness, and wide applicability of the method. The re-search indicates that this approach can validly improve the performance of text classification and pro-vide new solutions for natural language processing related tasks in long text and multi-category scenar-ios.