ClathPLM: Deep multi-view feature extraction with CNN and attention enhances clathrin protein identification.

To better understand and evaluate the GNSS positioning performance, it is convenient to adopt corresponding measures to reduce the impact of errors on positioning. A GNSS positioning error estimation scheme based on an improved multi-layer perceptron model is proposed. The multi-head attention mechanism and gating operation are integrated into the multi-layer perceptron model to adaptively select and filter features, enhancing the model’s ability to understand input features. First, the original positioning error of the satellite is obtained through the Kalman filter positioning method. The data are then preprocessed to extract available features. Finally, the features are input into the constructed model for training and testing to obtain the estimated positioning error value. Two types of comparative experiments were completed. The performance of the presented model is evaluated by the root mean square error. Experimental results show that the proposed method performs well in terms of performance indicators, and has obvious advantages over other state-of-the-art methods. In particular, the root mean square error of the presented method in the first dataset is 0.239 m, which is 39.2% and 17% lower than the current state-of-the-art long short-term memory network and convolutional neural network, respectively. The presented method can provide higher-precision estimated values for studying the GNSS positioning error estimation problem.

To overcome the limitations of existing deep learning-based image recognition approaches for diagnosing the operating conditions of pumping wells–namely, slow processing speed and high parameter complexity–this paper proposes a lightweight diagnostic method based on a one-dimensional convolutional neural network (1D CNN). Unlike traditional image-based techniques, the proposed method eliminates the need to convert raw data into dynamometer card images, thereby reducing computational overhead and significantly decreasing the number of model parameters. The architecture integrates 1D CNN with a bidirectional long short-term memory (BiLSTM) network to effectively capture both local patterns and global temporal dependencies inherent in different operating states of pumping wells. Additionally, a multi-head attention (MHA) mechanism is incorporated to enhance the representation of critical time points, thereby improving the model's sensitivity to key features and its overall classification performance. Experiments conducted on an augmented dataset–generated via a simulation-based algorithm–demonstrate that the proposed 1DCNN-BiLSTM-MHA model achieves a classification accuracy of 99.22%. Experimental results demonstrate that the proposed model possesses strong feature learning capabilities. Comparative analysis with several baseline models highlights its superior diagnostic performance and reliability, along with reduced parameter count and improved computational speed.

How to enable the computer to accurately analyze the emotional information and story background of characters in Qin opera is a problem that needs to be studied. To promote the artistic inheritance and cultural emotion color dissemination of Qin opera, an emotion analysis model of Qin opera based on attention residual network (ResNet) is presented. The neural network is improved and optimized from the perspective of the model, learning rate, network layers, and the network itself, and then multi-head attention is added to the ResNet to increase the recognition ability of the model. The convolutional neural network (CNN) is optimized from the internal depth, and the fitting ability and stability of the model are enhanced through the ResNet model. Combined with the attention mechanism, the expression of each weight information is strengthened. The multi-head attention mechanism is introduced in the model and a multi-head attention ResNet, namely, MHAtt_ResNet, is proposed. The network structure can effectively identify the features of the spectrogram, improve the weight information of spectrogram features, and deepen the relationship between distant information in long-time series. Through experiments, the proposed model has high emotional classification accuracy for Qin opera, and with the increase of the number of data sets, the model will train a better classification effect.

Generally, emotions in short videos are transmitted through characters behaviour and text content in the video. The mainstream recognition model is based on a convolutional neural network. However, its processing ability for video data is limited, its recognition accuracy is not high enough, and it is limited in processing complex text. To solve these problems, a new text recognition emotion classification model is designed by improving the text feature fusion module on the basis of the convolutional neural network model. Moreover, a new human behaviour recognition emotion classification model is designed by introducing multi-head attention mechanisms on the basis of a 3D convolutional neural network model. These results confirmed that the accuracy of the improved text recognition model was around 75%, while the original convolutional model's average accuracy was only about 67%. The 3D convolutional model under the multi-head attention mechanism had the highest recognition accuracy, with a recognition accuracy of 94.5% on the UCF101 database, which was 12 to 39 percentage points higher than the model under other attention mechanisms. The improved convolutional network model for short video text classification and behaviour recognition has more advantages than traditional models. These research results have certain value for classification models and can serve as technical references.

As one of the core tasks of natural language processing technology, text classification methods general-ly face the problems of insufficient global semantic capture and limited feature focusing ability when processing long texts or complex semantics. To address this issue, a deep learning model that integrates improved convolutional neural networks, unbalanced bidirectional long short-term memory networks, and multi-head attention mechanisms is proposed. Utilizing an improved bidirectional long short-term memory network to capture global semantic information, while dynamically focusing on key features through a multi head attention mechanism to enhance the model's adaptability to classification tasks. The performance of the model is validated through experiments on AG News (short text) and IMDb (long text) datasets. The results show that in short text classification, the proposed method has an accu-racy rate of 96% and a classification error rate of only 1.46%. In the task of long text classification, the method proposed in the study has a product under the curve of 0.98. In adversarial attack testing, the accuracy rates of adversarial samples generated by different methods are 92.85% and 90.63%, respec-tively, with the lowest robustness degradation rates of 3.72% and 5.49%, respectively. In cross domain generalization testing, it shows the least classification errors and superior cross domain adaptability. These results validate the high performance, robustness, and wide applicability of the method. The re-search indicates that this approach can validly improve the performance of text classification and pro-vide new solutions for natural language processing related tasks in long text and multi-category scenar-ios.

Sequential attention mechanism for weakly supervised video anomaly detection

Modeling accurate text recognizers for scanned Devanagari scripts is a challenging task, primarily due to the ingrained complexities of characters, vowels, conjuncts, and modifiers in the word formation of the script. This paper presents an attention-based convolutional recurrent neural network (CRNN) for improving the text recognition accuracy of the Devanagari script images. The proposed model is an encoder–decoder network with an attention mechanism. The encoder is a convolutional neural network (CNN), which extracts the significant and high-level features from the input images. The encoder also generates the extracted features in an ordered sequence. The decoder uses a bidirectional long short-term (BLSTM) network with an attention mechanism, followed by a connectionist temporal classification (CTC) layer to predict the text in images. The attention mechanism enables the decoder to selectively exploit the high-level features of the encoder congruently. We have trained, validated, and tested the performance of the proposed model on three datasets, viz. publicly available IIIT-H scene image dataset, synthetic dataset, and real dataset curated during this work. We have also compared the proposed model results with work already attempted in the area for the three datasets. We found that the performance of our proposed model outdoes the outcomes obtained from other models on the three datasets.

FeO content of sintered ore is an important reference index for measuring the performance of sintered ore. It significantly impacts the ironmaking process, iron quality, and energy consumption. Aiming at the current problem of delayed and poor accuracy of sintered ore FeO content detection results, this article proposes a hybrid network model that incorporates improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN), convolutional neural network (CNN), bidirectional long short-term memory (BiLSTM), and attention mechanism (AM) for FeO content prediction. First, the FeO content time series were decomposed using ICEEMDAN to obtain sub-layers with different frequencies. Then, the features with higher correlation with FeO content were selected by feature selection as model inputs, followed by predicting the decomposition sequences of FeO content using CNN-BiLSTM-AM for the feature-selected variables, respectively. Finally, all predicted sublayer predictions were reconstructed into the final prediction by summation. The proposed model effectively captures the essence of the sequence through the ICEEMDAN algorithm, extracts deep features from FeO content data using CNN, captures contextual information through BiLSTM, and enhances feature extraction capability with AM. The experimental results show that the collaboration of CNN, BiLSTM, and AM effectively enhances the modelling capability of the model and significantly improves the prediction accuracy. Additionally, the ICEEMDAN decomposition algorithm is employed to enhance the prediction performance further, offering advantages over other decomposition techniques. The MAE, MAPE, RMSE, RRMSE, and R² of the new ICEEMDAN-CNN-BiLSTM-AM (ICBA) model are 0.0751, 0.846%, 0.0937, 1.0500%, and 0.9646, respectively, demonstrating a significant improvement in prediction accuracy and outperforming the relevant comparison models.

COVID-19 fake news detection: A hybrid CNN-BiLSTM-AM model

Efficient multimodel method based on transformers and CoAtNet for Alzheimer's diagnosis

To satisfy the need to accurately monitor emotional stress, this paper explores the effectiveness of the attention mechanism based on the deep learning model CNN (Convolutional Neural Networks)-BiLSTM (Bi-directional Long Short-Term Memory) As different attention mechanisms can cause the framework to focus on different positions of the feature map, this discussion adds attention mechanisms to the CNN layer and the BiLSTM layer separately, and to both the CNN layer and BiLSTM layer simultaneously to generate different CNN–BiLSTM networks with attention mechanisms. ECG (electrocardiogram) data from 34 subjects were collected on the server platform created by the Institute of Psychology of the Chinese Academy of Science and the researches. It verifies that the average accuracy of CNN–BiLSTM is up to 0.865 without any attention mechanism, while the highest average accuracy of 0.868 is achieved using the CNN–attention–based BiLSTM.

Deep learning theory demonstrates its great power in the complicated data processing field. As a prevalent deep learning technology, convolutional neural network (CNN) has achieved some successful applications in the field of fault detection and classification in the complicated industrial systems because of its strong nonlinear feature extraction capability. However, the basic CNN views all the features equally and can not highlight these features which play a more important role in the fault classification procedure. To deal with this problem, this paper proposes an improved CNN model, called multi-head attention CNN (MACNN), to distinguish the importance of different features for better fault classification performance. In the MACNN framework, the raw training data are firstly analyzed by the multiple convolutional layers for feature extraction. Then, the features obtained by the convolution layers are used as the input of the multi-head attention (MA) layers. The introduction of the MA mechanism can help exploit more meaningful information by emphasizing the important features. Lastly, the softmax classifier is utilized to classify the fault patterns. To test the method performance, one case study on the benchmark Tennessee Eastman industrial system is performed and the results show that MACNN gives the better fault classification results than the traditional feedforward neural network (FNN) and CNN models.

Identifying the massage techniques of the masseuse is a prerequisite for guiding robotic massage. It is difficult to recognize multiple consecutive massage maps with a time series for current human action recognition algorithms. To solve the problem, a method combining a convolutional neural network, long-term neural network, and attention mechanism is proposed to identify the massage techniques in this paper. First, the pressure distribution massage map is collected by a massage glove, and the data are enhanced by the conditional variational auto-encoder. Then, the features of the massage map group in the spatial domain and timing domain are extracted through the convolutional neural network and the long- and short-term memory neural network, respectively. The attention mechanism is introduced into the neural network, giving each massage map a different weight value to enhance the network extraction of data features. Finally, the massage haptic dataset is collected by a massage data acquisition system. The experimental results show that a classification accuracy of 100% is achieved. The results demonstrate that the proposed method could identify sequential massage maps, improve the network overfitting phenomenon, and enhance the network generalization ability effectively.

People deploy top-down, goal-directed attention to accomplish tasks, such as finding lost keys. By tuning the visual system to relevant information sources, object recognition can become more efficient (a benefit) and more biased toward the target (a potential cost). Motivated by selective attention in categorisation models, we developed a goal-directed attention mechanism that can process naturalistic (photographic) stimuli. Our attention mechanism can be incorporated into any existing deep convolutional neural networks (DCNNs). The processing stages in DCNNs have been related to ventral visual stream. In that light, our attentional mechanism incorporates top-down influences from prefrontal cortex (PFC) to support goal-directed behaviour. Akin to how attention weights in categorisation models warp representational spaces, we introduce a layer of attention weights to the mid-level of a DCNN that amplify or attenuate activity to further a goal. We evaluated the attentional mechanism using photographic stimuli, varying the attentional target. We found that increasing goal-directed attention has benefits (increasing hit rates) and costs (increasing false alarm rates). At a moderate level, attention improves sensitivity (i.e. increases d^{prime }) at only a moderate increase in bias for tasks involving standard images, blended images and natural adversarial images chosen to fool DCNNs. These results suggest that goal-directed attention can reconfigure general-purpose DCNNs to better suit the current task goal, much like PFC modulates activity along the ventral stream. In addition to being more parsimonious and brain consistent, the mid-level attention approach performed better than a standard machine learning approach for transfer learning, namely retraining the final network layer to accommodate the new task.

Bearings play a crucial role in rotating machinery, and their failure significantly impacts performance and safety. Early fault detection is challenging due to weak signals, especially under variable operating conditions. Deep learning techniques, such as Convolutional Neural Networks (CNN) and attention mechanisms, have shown great potential in improving fault detection accuracy in recent years. However, handling complex fault patterns and extracting early fault features remain challenging. To address these issues, this paper proposes a new method that combines Temporal Convolutional Networks (TCN), CNN, and Multi-Head Attention. TCN captures short-term dependencies, CNN extracts long-term features, and Multi-Head Attention enhances focus on key fault-related features. The model was tested on the Case Western Reserve University (CWRU) bearing fault dataset and compared with baseline models, showing a significant improvement in accuracy, achieving 99.3%. This study provides valuable insights for handling complex and multi-fault scenarios, offering essential implications for industrial applications.