Facial Expression Recognition method based on residual network and attention mechanism

Facial expression methods play a vital role in human-computer interaction and other fields, but there are factors such as occlusion, illumination, and pose changes in wild facial recognition, as well as category imbalances between different datasets, that result in large variations in recognition rates and low accuracy rates for different categories of facial expression datasets. This study introduces RCL-Net, a method of recognizing wild facial expressions that is based on an attention mechanism and LBP feature fusion. The structure consists of two main branches, namely the ResNet-CBAM residual attention branch and the local binary feature (LBP) extraction branch (RCL-Net). First, by merging the residual network and hybrid attention mechanism, the residual attention network is presented to emphasize the local detail feature information of facial expressions; the significant characteristics of facial expressions are retrieved from both channel and spatial dimensions to build the residual attention classification model. Second, we present a locally improved residual network attention model. LBP features are introduced into the facial expression feature extraction stage in order to extract texture information on expression photographs in order to emphasize facial feature information and enhance the recognition accuracy of the model. Lastly, experimental validation is performed using the FER2013, FERPLUS, CK+, and RAF-DB datasets, and the experimental results demonstrate that the proposed method has superior generalization capability and robustness in the laboratory-controlled environment and field environment compared to the most recent experimental methods.

Brain-computer interfaces (BCI) allow for the brain to communicate with electronic devices. Concerning the BCI paradigms, motor imagery uses brain signals to decode an imagined movement. However, this is a hard task given the low signal-to-noise ratio. Usually, the main steps in BCI models are pre-processing, feature extraction, and classification. In recent years, Convolutional Neural Networks (CNNs) have been gaining relevance in several areas of science due to their feature extraction, translation invariance, and parameter sharing capa-bilities. Another, more recent way of feature extraction is using attention mechanisms, which are layers of neural networks based on human attention and have the ability to highlight important features. A variation of the attention mechanism is the Convolutional Block Attention Module, which combines the CNN structure with the attention mechanism. In this work, we propose a new model that joins the core architecture of EEGNet, a compact CNN widely used in the literature, with the Convolutional Block Attention Module and residual connections. The residual connections were introduced to lower data degradation throughout the model. The results highlight the residual connection's importance for the performance of the model. The proposed model obtained a kappa result 5.2% better than the EEGNet with a p-value less than 0.01 on BCI Competition IV dataset 2a, which is a well-known dataset for Motor Imagery. Furthermore, the proposal was better than EEGNet for most subjects and had the best-worst case.

MRI image classification of brain tumors is critical for accurate and early diagnosis. New developments in deep learning have revealed that inserting attention mechanisms into convolutional neural networks can greatly improve classification performance. This work assesses the effectiveness of Squeeze-and-Excitation (SE) and Convolutional Block Attention Module (CBAM) integrated with the ResNet50 model for brain tumor classification. The SE module recalibrates channel-wise feature responses, while CBAM enhances feature representation by combining channel and spatial attention. The SE and CBAM are combined with ResNet50, greatly increasing classification accuracy, precision, and recall when compared to the basic model, according to experimental results on MRI datasets. The suggested model employs attention mechanisms to focus valuable information selectively and suppress irrelevant information. The experiments on publicly available brain tumor datasets display great improvements over basic CNN models, with precision, recall, accuracy, as well as F1 score at 0.9914, 0.9903, 0.9945, and 0.9908, respectively. From these results, the importance of attention mechanisms in deep learning models for medical imaging is highlighted, which suggests that SE and CBAM modules can be available as more dependable and effective instruments for brain tumor classification in clinical settings. Future studies should investigate transformer-based and hybrid attention techniques to enhance automated brain tumor categorization.

Motivated by the increasing popularity and success of attention mechanisms, we observe that popular convolutional attention models like Squeeze-and-Excite (SE) and Convolutional Block Attention Module (CBAM) rely on expensive multi-layer perceptron (MLP) layers. These MLP layers significantly increase computational complexity, making such models less applicable to 3D image contexts, where the data dimensionality and computational costs are inherently higher. In 3D medical imaging, such as pulmonary CT scans, efficient processing is crucial due to the large volume of data. Traditional 2D attention mechanisms, when generalized to 3D, increase the computational load, creating a demand for more efficient attention mechanisms that can operate effectively within a 3D tasks. In this work, we investigate the possibility of incorporating fully convolutional attention within 3D context. Particularly, we present two 3D fully convolutional attention blocks in which we demonstrate effectiveness within the 3D context. We demonstrate effectiveness of the proposed method using the pulmonary CT scans for 3D lung nodule detection. Building on existing 3D nodule detection methods, we present AttentNet, an automated lung nodule detection framework from CT images that performs detection as an ensemble of two stages, candidate proposal stage (Region Proposal Network), in which a high number of candidates is produced, and a false positive reduction stage to reduce the number of false alarms. Throughout our work, we compare the proposed 3D convolutional attention blocks to a number of popular 2D convolutional attention mechanisms by generalizing them to 3D modules, including SE units and CBAM channel attention and spatial attention units. We also compare these to Transformer self-attention units. For the False positive reduction stage, we incorporate a joint analysis approach that considers the variable nodule morphology by aggregating spatial information from different contextual levels. We use LUNA-16 lung nodule detection dataset to demonstrate the benefits of using the proposed fully convolutional attention blocks comparing to baseline popular lung nodule detection methods where no attention is used. It is worth noting that our work does not aim at achieving state-of-the-art results in the lung nodule detection task, rather to demonstrate the benefits of incorporating fully convolutional attention within a 3D context.

This study focuses on the optimization and practice of Convolutional Neural Networks (CNN) in image classification tasks. A deep neural network model was constructed by introducing the CBAM (Convolutional Block Attention Module) attention mechanism and residual structure. This model combines the stability of residual networks during training with the advantages of CBAM in feature extraction, aiming to improve image classification performance. The experiment used the CIFAR-10 dataset and achieved a testing accuracy of 86% through optimization methods such as data augmentation, batch normalization, reasonable control of the number of fully connected layers, and exponential decay learning rate strategy. The study also explored the effects of data augmentation, batch normalization layer utility, fully connected layer quantity control, residual structure effectiveness, and CBAM attention mechanism on model performance, providing valuable insights for optimizing CNN in image classification tasks.

Acquiring the growth information of tobacco plants quickly and accurately is an important prerequisite for their daily management. In view of the problem that the convolutional neural network is easily affected by background information, this study introduces the Convolutional Block Attention Module into the U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2-</sup> Net. This method is mainly composed of an encoder and a decoder. Before the feature map is decoded, it will go through the Convolutional Block Attention Module to filter more important features and inhibit the invalid features. In the decoder, in order to fuse the features of objects in different scales, the feature maps in the decoder are up-sampled and then concatenated. This information aggregation solves the problem of difficult object detection at different scales to some extent. The results of this method are compared with four current mainstream deep learning models(U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net, DeeplabV3+, PSP-Net, U-Net). The result shows: 1) The U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net network based on the Convolutional Block Attention Module has the highest extraction accuracy for the UAV visible light remote senseing of tobacco(accuracy was 98.53%, recall was 96.00%,F1-Score was 96.38%, kappa coefficient was 0.95, IOU was 93.02%); 2)In the extraction of the three test areas, the extraction value of this method is highly consistent with the label area value(the area ratios of a total of 13 plants in the three regions were kept between 90% and 100%),the effect is much higher than that of DeeplabV3+ and PSP-Net, and slightly higher than the extraction effect of U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net and U-Net. 3)In the segmentation of three typical canopy closure regions of A, B, and C, most of the accuracy factors in different regions can reach 94% and above. It can be seen that the method in this study has certain advantages in feature extraction and has certain value in practical application.

Wind turbine rolling bearings are crucial components for ensuring the reliability and stability of wind power systems. Their failure can lead to significant economic losses and equipment downtime. Therefore, the accurate diagnosis of bearing faults is of great importance. Although existing deep learning fault diagnosis methods have achieved certain results, they still face limitations such as inadequate feature extraction capabilities, insufficient generalization to complex working conditions, and ineffective multi-scale feature capture. To address these issues, this paper proposes an advanced fault diagnosis method named the two-stream feature fusion convolutional neural network (TSFFResNet-Net). Firstly, the proposed method combines the advantages of one-dimensional convolutional neural networks (1D-ResNet) and two-dimensional convolutional neural networks (2D-ResNet). It transforms one-dimensional vibration signals into two-dimensional images through the empirical wavelet transform (EWT) method. Then, parallel convolutional kernels in 1D-ResNet and 2D-ResNet are used to extract multi-scale features, respectively. Next, the Convolutional Block Attention Module (CBAM) is introduced to enhance the network’s ability to capture key features by focusing on important features in specific channels or spatial areas. After feature fusion, CBAM is introduced again to further enhance the effect of feature fusion, ensuring that the features extracted by different network branches can be effectively integrated, ultimately providing more accurate input features for the classification task of the fully connected layer. The experimental results demonstrate that the proposed method outperforms other traditional methods and advanced convolutional neural network models on different datasets. Compared with convolutional neural network models such as LeNet-5, AlexNet, and ResNet, the proposed method achieves a significantly higher accuracy on the test set, with a stable accuracy of over 99%. Compared with other models, it shows better generalization and stability, effectively improving the overall performance of rolling bearing vibration signal fault diagnosis. The method provides an effective solution for the intelligent fault diagnosis of wind turbine rolling bearings.

Because of its long running time, complex working environment, and for other reasons, a gear is prone to failure, and early failure is difficult to detect by direct observation; therefore, fault diagnosis of gears is very necessary. Neural network algorithms have been widely used to realize gear fault diagnosis, but the structure of the neural network model is complicated, the training time is long and the model is not easy to converge. To solve the above problems and combine the advantages of the ResNeXt50 model in the extraction of image features, this paper proposes a gearbox fault detection method that integrates the convolutional block attention module (CBAM). Firstly, the CBAM is embedded in the ResNeXt50 network to enhance the extraction of image channels and spatial features. Secondly, the different time–frequency analysis method was compared and analyzed, and the method with the better effect was selected to convert the one-dimensional vibration signal in the open data set of the gearbox into a two-dimensional image, eliminating the influence of the redundant background noise, and took it as the input of the model for training. Finally, the accuracy and the average training time of the model were obtained by entering the test set into the model, and the results were compared with four other classical convolutional neural network models. The results show that the proposed method performs well both in fault identification accuracy and average training time under two working conditions, and it also provides some references for existing gear failure diagnosis research.

Recently, multi-level feature networks have been extensively used in instance segmentation. However, because not all features are beneficial to instance segmentation tasks, the performance of networks cannot be adequately improved by synthesizing multi-level convolutional features indiscriminately. In order to solve the problem, an attention-based feature pyramid module (AFPM) is proposed, which integrates the attention mechanism on the basis of a multi-level feature pyramid network to efficiently and pertinently extract the high-level semantic features and low-level spatial structure features; for instance, segmentation. Firstly, we adopt a convolutional block attention module (CBAM) into feature extraction, and sequentially generate attention maps which focus on instance-related features along the channel and spatial dimensions. Secondly, we build inter-dimensional dependencies through a convolutional triplet attention module (CTAM) in lateral attention connections, which is used to propagate a helpful semantic feature map and filter redundant informative features irrelevant to instance objects. Finally, we construct branches for feature enhancement to strengthen detailed information to boost the entire feature hierarchy of the network. The experimental results on the Cityscapes dataset manifest that the proposed module outperforms other excellent methods under different evaluation metrics and effectively upgrades the performance of the instance segmentation method.

In the intelligent tomato-picking scenario, challenges such as insufficient accuracy in recognizing the growth pose of target tomatoes and inaccurate positioning of picking and grasping points have led to low efficiency in automated picking. To address these issues, this paper introduces an object detection optimization model based on Yolov8s, termed YOLOv8S-ECC. The model focuses on “Judging tomato pose by the spatial vector of the relative position between the calyx and the center point of the fruit,” aiming to enhance high-precision positioning of both the tomato calyx and fruit, thereby laying the groundwork for subsequent pose judgment and picking point positioning. We have integrated the ECA (Efficient Channel Attention) and Coordinate attention mechanisms into the Backbone network and introduced the CBAM (Convolutional Block Attention Module) attention mechanism into the Neck network. The combined effect of these attention mechanisms effectively overcomes the recognition challenges posed by the calyx’s color texture, which closely resembles the environment. This integration has also enhanced the model’s robustness in complex field environments. Test results indicate significant improvements: the accuracy rate, recall rate, and mAP@50 for detecting tomato fruits and calyces are 81.7% and 87.5%, 92.7% and 85.9%, and 89.7% and 91.3%, respectively, compared to the original model. By encapsulating the algorithm and integrating it with the picking robot, tests in a simulated environment (different lighting conditions and foliage occlusion situations) show picking success rates of 93.02%, with an average picking operation time of 14.2 ± 0.855 s, including an image recognition and processing time of 0.035 s. This research offers an effective technical solution for high-precision visual perception and pose judgment in fruit and vegetable picking robots, contributing to improved quality in tomato industry picking operations.

To reduce the chance of being infected by the COVID-19, wearing masks correctly when entering and leaving public places has become the most feasible and effective ways to prevent the spread of the virus. It is a concern to how to quickly and accurately detect whether a face is worn a mask correctly while reduce missed detection and false detection in practical applied scenarios. In this paper, an improved algorithm is proposed based on the YOLO-v4 algorithm. The attention mechanism module is added to the appropriate network level to enhance the key feature points of face wearing masks and suppress useless information. Apart from that, three attention mechanism modules are added to different layers of the YOLO-v4 network for ablation experiments, including CBAM (convolutional block attention module), SENet (squeeze-and-excitation networks) and CANet (coordinate attention networks). The path-aggregation network and feature pyramid are used to extract features from images. Two network models were compared and improved in the experiment, and it is found that adding the dual-channel attention mechanism CBAM before the three YOLO heads of YOLOv4 and in the neck network had better detection performance than the single channel attention mechanism SENet and the coordinated attention mechanism CANet. The experimental results show that when the attention module CBAM and the YOLO-v4 model are integrated, the accuracy of the selected MAFA + WIDER Face dataset reaches the highest value of 93.56%, which is 4.66% higher than that of the original YOLO-v4.

As power grid structure becomes increasingly complex, the impact of the operation status of key substation equipment on the safe and stable operation of the power grid is gradually increasing. Aiming at the problems of dense targets and occlusion recognition difficulties in the complex background of substations, an improved you-only-look-once (YOLO) v4 insulator defect detection algorithm is proposed. A large number of insulator and defect datasets were collected from the Internet. We use MobileNet-v2 to replace the backbone network of YOLOv4, and in view of the problem that the accuracy of YOLOv4 target detection model is not high for small targets or dense targets, convolutional block attention module (CBAM) attention mechanism is added to the backbone network to enhance the feature extraction ability, and ultra-lightweight subspace attention module (ULSAM) is added to the detection head to learn the cross-channel information. Experiments based on InDataset show that the insulator defect detection algorithm based on deep learning proposed achieves a high detection accuracy and detection speed; the mean average precision of the improved YOLOv4 model is 87.48%, and the recall rate of insulator defects reaches 79.84%, which meets the requirements of robustness and accuracy of insulator defect detection in real scenarios.

Focusing on the problems of complex structure and low feature extraction efficiency that exist in some traditional neural network algorithms, an improved convolutional neural network, combined with a convolutional attention mechanism, is proposed to im- prove the network feature extraction efficiency and reduce the network complexity. First, the convolutional block attention module (CBAM) was introduced, which can strengthen the extraction of useful features from the data, thereby improving the network feature extraction efficiency. Second, the residual structure was optimized, and the original multi-layer convolution was simplified to a single-layer convolution, which reduces the complexity of the model and improves the diagnostic efficiency. Then, the Rectified Linear Unit (ReLU) activation function in the model was replaced with the better-performing Exponential Linear Unit (ELU) activation function. Finally, the overall structure of the ResNet model was optimized. The CBAM-ResNet network model was based on the traditional ResNet18 model and consists of a CBAM module and the optimized ResNet18 model. Finally, experiments were performed using sensor data collected from rolling bearing test benches. The experimental results show that the feature extraction capability and efficiency of the new model are better than those of several other classic deep network models and the original network model, and this reduces the time complexity and model size while also maintaining a high accuracy rate.

Nowcasting is a critical technology for disaster prevention and mitigation, and the accuracy of radar echo extrapolation directly impacts forecasting performance. In most deep learning-based models, accurately predicting heavy precipitation remains a challenging task. Focusing on the region of China, this study proposes an improved model based on residual and attention mechanisms—RA-UNet—for precipitation nowcasting with a lead time of 3 h. The model introduces the residual neural network (ResNet) and the convolutional block attention module (CBAM) to integrate multi-scale features into the U-Net encoder–decoder architecture, enhancing its ability to capture the spatiotemporal evolution of precipitation systems. Meanwhile, depthwise separable convolutions are employed to replace conventional convolutions, significantly improving computational efficiency while preserving model performance. To evaluate the model’s performance, experiments were conducted using 6 min resolution radar echo data from China in 2024, with comparisons made against the optical flow (OF) method and the U-Net model. The experimental results show that RA-UNet demonstrates significant advantages in 3 h forecasting: its mean absolute error (MAE) is reduced by approximately 7%, the false alarm rate (FAR) decreases by about 20%, and it outperforms the comparison models in metrics such as the critical success index (CSI) and structural similarity index (SSIM). Notably, RA-UNet effectively mitigates intensity degradation in long-term forecasts, successfully predicting the trend of &gt;40 dBZ strong echo cores in two typical cases and significantly improving the premature dissipation problem of precipitation fields. This study provides a new approach to refined forecasting of complex precipitation systems, and future work will combine multi-source data fusion with physical constraint mechanisms to further enhance precipitation event prediction capabilities.

With the continuous development of deep learning technologies, fault prediction techniques based on various neural networks have been evolving. The deep learning modules based on U-Net residual networks have shown significant advantages in both learning efficiency and effectiveness. In this paper, we propose a deep learning model that integrates a 3D U-Net residual architecture, Convolutional Block Attention Module (CBAM), and Multi-scale Enhanced Global Attention (MEGA) module for automatic seismic fault detection and segmentation. This model can effectively handle complex 3D seismic data, fully exploiting both spatial and channel information, significantly improving the prediction accuracy for small faults, while only slightly increasing the computational cost.Firstly, the model uses the 3D U-Net as the backbone framework, where the residual blocks (BasicRes) extract features through multiple convolution layers. The CBAM module is incorporated to apply attention weighting, enhancing the model's ability to focus on critical information. The CBAM module combines channel attention and spatial attention, effectively adjusting the importance of feature maps from different dimensions, enabling the model to identify potential fault features in complex seismic data.Secondly, the MEGA module is introduced into the model, which further improves the model's feature representation ability by fusing multi-scale features and applying a global attention mechanism. By weighting global information, the MEGA module helps the model better capture key seismic fault features during feature fusion. This design allows the model to focus not only on local details but also to fully utilize the global contextual information in 3D data, thereby enhancing the accuracy of fault detection.After validation, the model achieved promising results in seismic fault detection tasks, automatically identifying and segmenting fault structures in seismic data. The accuracy was improved from 80% with the original 3D U-Net residual network to 85%-87%. This provides strong support for applications such as seismic exploration and subsurface imaging.Keywords: Seismic Fault Detection, 3D U-Net, Convolutional Block Attention Module (CBAM), Multi-scale Enhanced Global Attention (MEGA), Deep Learning