An Efficient Module for Instance Segmentation Based on Multi-Level Features and Attention Mechanisms

Brain-computer interfaces (BCI) allow for the brain to communicate with electronic devices. Concerning the BCI paradigms, motor imagery uses brain signals to decode an imagined movement. However, this is a hard task given the low signal-to-noise ratio. Usually, the main steps in BCI models are pre-processing, feature extraction, and classification. In recent years, Convolutional Neural Networks (CNNs) have been gaining relevance in several areas of science due to their feature extraction, translation invariance, and parameter sharing capa-bilities. Another, more recent way of feature extraction is using attention mechanisms, which are layers of neural networks based on human attention and have the ability to highlight important features. A variation of the attention mechanism is the Convolutional Block Attention Module, which combines the CNN structure with the attention mechanism. In this work, we propose a new model that joins the core architecture of EEGNet, a compact CNN widely used in the literature, with the Convolutional Block Attention Module and residual connections. The residual connections were introduced to lower data degradation throughout the model. The results highlight the residual connection's importance for the performance of the model. The proposed model obtained a kappa result 5.2% better than the EEGNet with a p-value less than 0.01 on BCI Competition IV dataset 2a, which is a well-known dataset for Motor Imagery. Furthermore, the proposal was better than EEGNet for most subjects and had the best-worst case.

Multi-style transfer and fusion of image’s regions based on attention mechanism and instance segmentation

To reduce the chance of being infected by the COVID-19, wearing masks correctly when entering and leaving public places has become the most feasible and effective ways to prevent the spread of the virus. It is a concern to how to quickly and accurately detect whether a face is worn a mask correctly while reduce missed detection and false detection in practical applied scenarios. In this paper, an improved algorithm is proposed based on the YOLO-v4 algorithm. The attention mechanism module is added to the appropriate network level to enhance the key feature points of face wearing masks and suppress useless information. Apart from that, three attention mechanism modules are added to different layers of the YOLO-v4 network for ablation experiments, including CBAM (convolutional block attention module), SENet (squeeze-and-excitation networks) and CANet (coordinate attention networks). The path-aggregation network and feature pyramid are used to extract features from images. Two network models were compared and improved in the experiment, and it is found that adding the dual-channel attention mechanism CBAM before the three YOLO heads of YOLOv4 and in the neck network had better detection performance than the single channel attention mechanism SENet and the coordinated attention mechanism CANet. The experimental results show that when the attention module CBAM and the YOLO-v4 model are integrated, the accuracy of the selected MAFA + WIDER Face dataset reaches the highest value of 93.56%, which is 4.66% higher than that of the original YOLO-v4.

Motivated by the increasing popularity and success of attention mechanisms, we observe that popular convolutional attention models like Squeeze-and-Excite (SE) and Convolutional Block Attention Module (CBAM) rely on expensive multi-layer perceptron (MLP) layers. These MLP layers significantly increase computational complexity, making such models less applicable to 3D image contexts, where the data dimensionality and computational costs are inherently higher. In 3D medical imaging, such as pulmonary CT scans, efficient processing is crucial due to the large volume of data. Traditional 2D attention mechanisms, when generalized to 3D, increase the computational load, creating a demand for more efficient attention mechanisms that can operate effectively within a 3D tasks. In this work, we investigate the possibility of incorporating fully convolutional attention within 3D context. Particularly, we present two 3D fully convolutional attention blocks in which we demonstrate effectiveness within the 3D context. We demonstrate effectiveness of the proposed method using the pulmonary CT scans for 3D lung nodule detection. Building on existing 3D nodule detection methods, we present AttentNet, an automated lung nodule detection framework from CT images that performs detection as an ensemble of two stages, candidate proposal stage (Region Proposal Network), in which a high number of candidates is produced, and a false positive reduction stage to reduce the number of false alarms. Throughout our work, we compare the proposed 3D convolutional attention blocks to a number of popular 2D convolutional attention mechanisms by generalizing them to 3D modules, including SE units and CBAM channel attention and spatial attention units. We also compare these to Transformer self-attention units. For the False positive reduction stage, we incorporate a joint analysis approach that considers the variable nodule morphology by aggregating spatial information from different contextual levels. We use LUNA-16 lung nodule detection dataset to demonstrate the benefits of using the proposed fully convolutional attention blocks comparing to baseline popular lung nodule detection methods where no attention is used. It is worth noting that our work does not aim at achieving state-of-the-art results in the lung nodule detection task, rather to demonstrate the benefits of incorporating fully convolutional attention within a 3D context.

Acquiring the growth information of tobacco plants quickly and accurately is an important prerequisite for their daily management. In view of the problem that the convolutional neural network is easily affected by background information, this study introduces the Convolutional Block Attention Module into the U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2-</sup> Net. This method is mainly composed of an encoder and a decoder. Before the feature map is decoded, it will go through the Convolutional Block Attention Module to filter more important features and inhibit the invalid features. In the decoder, in order to fuse the features of objects in different scales, the feature maps in the decoder are up-sampled and then concatenated. This information aggregation solves the problem of difficult object detection at different scales to some extent. The results of this method are compared with four current mainstream deep learning models(U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net, DeeplabV3+, PSP-Net, U-Net). The result shows: 1) The U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net network based on the Convolutional Block Attention Module has the highest extraction accuracy for the UAV visible light remote senseing of tobacco(accuracy was 98.53%, recall was 96.00%,F1-Score was 96.38%, kappa coefficient was 0.95, IOU was 93.02%); 2)In the extraction of the three test areas, the extraction value of this method is highly consistent with the label area value(the area ratios of a total of 13 plants in the three regions were kept between 90% and 100%),the effect is much higher than that of DeeplabV3+ and PSP-Net, and slightly higher than the extraction effect of U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net and U-Net. 3)In the segmentation of three typical canopy closure regions of A, B, and C, most of the accuracy factors in different regions can reach 94% and above. It can be seen that the method in this study has certain advantages in feature extraction and has certain value in practical application.

With the continuous development of deep learning technologies, fault prediction techniques based on various neural networks have been evolving. The deep learning modules based on U-Net residual networks have shown significant advantages in both learning efficiency and effectiveness. In this paper, we propose a deep learning model that integrates a 3D U-Net residual architecture, Convolutional Block Attention Module (CBAM), and Multi-scale Enhanced Global Attention (MEGA) module for automatic seismic fault detection and segmentation. This model can effectively handle complex 3D seismic data, fully exploiting both spatial and channel information, significantly improving the prediction accuracy for small faults, while only slightly increasing the computational cost.Firstly, the model uses the 3D U-Net as the backbone framework, where the residual blocks (BasicRes) extract features through multiple convolution layers. The CBAM module is incorporated to apply attention weighting, enhancing the model's ability to focus on critical information. The CBAM module combines channel attention and spatial attention, effectively adjusting the importance of feature maps from different dimensions, enabling the model to identify potential fault features in complex seismic data.Secondly, the MEGA module is introduced into the model, which further improves the model's feature representation ability by fusing multi-scale features and applying a global attention mechanism. By weighting global information, the MEGA module helps the model better capture key seismic fault features during feature fusion. This design allows the model to focus not only on local details but also to fully utilize the global contextual information in 3D data, thereby enhancing the accuracy of fault detection.After validation, the model achieved promising results in seismic fault detection tasks, automatically identifying and segmenting fault structures in seismic data. The accuracy was improved from 80% with the original 3D U-Net residual network to 85%-87%. This provides strong support for applications such as seismic exploration and subsurface imaging.Keywords: Seismic Fault Detection, 3D U-Net, Convolutional Block Attention Module (CBAM), Multi-scale Enhanced Global Attention (MEGA), Deep Learning

In recent years, attention mechanisms have shown great potential in various computer vision tasks. However, most existing methods focus on developing more complex attention modules for better performance, which inevitably increases the complexity of the model. To overcome performance and complexity tradeoffs, this paper proposes efficient spatial and channel attention (ESCA), a symmetric, comprehensive, and efficient attention module. By analyzing squeeze-and-excitation (SE), convolutional block attention module (CBAM), coordinate attention (CA), and efficient channel attention (ECA) modules, we abandon the dimension-reduction operation of SE module, verify the negative impact of global max pooling (GMP) on the model, and apply a local cross-channel interaction strategy without dimension reduction to learn attention. We not only care about the channel features of the image, we also care about the spatial location of the target on the image, and we take into account the effectiveness of channel attention, so we designed the symmetric ESCA module. The ESCA module is effective, as demonstrated by its application in the ResNet-50 classification benchmark. With 26.26 M parameters and 8.545 G FLOPs, it introduces a mere 0.14% increment in FLOPs while achieving over 6.33% improvement in Top-1 accuracy and exceeding 3.25% gain in Top-5 accuracy. We perform image classification and object detection tasks on ResNet, MobileNet, YOLO, and other architectures on popular datasets such as Mini ImageNet, CIFAR-10, and VOC 2007. Experiments show that ESCA can achieve great improvement in model accuracy at a very small cost, and it performs well among similar models.

MRI image classification of brain tumors is critical for accurate and early diagnosis. New developments in deep learning have revealed that inserting attention mechanisms into convolutional neural networks can greatly improve classification performance. This work assesses the effectiveness of Squeeze-and-Excitation (SE) and Convolutional Block Attention Module (CBAM) integrated with the ResNet50 model for brain tumor classification. The SE module recalibrates channel-wise feature responses, while CBAM enhances feature representation by combining channel and spatial attention. The SE and CBAM are combined with ResNet50, greatly increasing classification accuracy, precision, and recall when compared to the basic model, according to experimental results on MRI datasets. The suggested model employs attention mechanisms to focus valuable information selectively and suppress irrelevant information. The experiments on publicly available brain tumor datasets display great improvements over basic CNN models, with precision, recall, accuracy, as well as F1 score at 0.9914, 0.9903, 0.9945, and 0.9908, respectively. From these results, the importance of attention mechanisms in deep learning models for medical imaging is highlighted, which suggests that SE and CBAM modules can be available as more dependable and effective instruments for brain tumor classification in clinical settings. Future studies should investigate transformer-based and hybrid attention techniques to enhance automated brain tumor categorization.

As the core technologies represented by Internet of Things (IoT) technology and computer vision technology booming, object detection algorithms based on deep learning have received high attention in various fields. Object detection is not only a core problem in the field of computer vision, but also a prerequisite and basis for many computer vision tasks, and has important applications in the fields of autonomous driving and video surveillance. The task of object detection is to find all the targets of interest in an image and determine their category and location. Object detection includes two tasks: object classification and object localization. However, object classification focuses on the most discriminative part of the feature map, and object localization requires a feature map focusing on the whole region of the object. Therefore, in order to improve the detection accuracy, we propose a CBAM-RetinaNet network model.CBAM (Convolutional Block Attention Module) is a lightweight convolutional attention module, which combines channel and spatial attention mechanism modules. The channel attention module focuses on the meaningful information in the input image, which would lead to an increase in the accuracy of object classification. The spatial attention module focuses on the location information of the object, which may improve the accuracy of object localization. The experiment results show that the proposed achieves better object detection performance on the PASCAL VOC2007 dataset, and the accuracy is 2% higher than that of the RetinaNet model, which has good experimental results.

Weeds are one of the primary factors that reduce crop productivity by competing for nutrients and water, causing the plant to lose weight and resulting in reduced grain yield. Traditional agricultural practices often rely on uniform herbicide application, which can contaminate soil and raise costs. On agricultural land, selective weed treatment are an efficient and cost-effective way to control weeds that require a deep learning-based crop and weed segmentation system. Many existing crop and weed segmentation research works focus on achieving precise crop and weed segmentation results, rather than building lightweight models to deploy on edge devices. To attain this, we develop an effective and efficient convolutional neural network, namely the Concatenated Attention U-Net with Convolutional Block Attention Module (CAUC). By integrating Linear Concatenated Blocks (LCB), Attention Gate (AG) connections, and Convolutional Block Attention Module (CBAM), the proposed model efficiently utilizes feature maps among its architectural components to achieve superior performance. Depth-wise convolution layers and 1 × 1 convolution layers in LCBs reduce computational complexity. To enable the proposed model to identify the weed portions in multiple crop fields, we integrated three datasets in this research work, namely the Crop/Weed Field Image Dataset (CWFID), Sugar Beet, and Sunflower datasets. Experimental results on carrot, sugar beet, and sunflower crop datasets demonstrate high Accuracy (99.09%), MIoU (81.02%), and F1-score (99.06%), with a modest model size (5.6 MB) and computational parameters (0.377 million). We developed a lightweight computer vision application (13.7 MB) to demonstrate the model’s efficacy on low-computational devices.

Lotus seedpod maturity detection and segmentation in pond environments play a significant role in yield prediction and picking pose estimation for lotus seedpods. However, it is a great challenge to accurately detect and segment lotus seedpods due to insignificant phenotypic differences between the adjacent maturity, changing illumination, overlap, and occlusion of lotus seedpods. The existing research pays attention to lotus seedpod detection while ignoring maturity detection and segmentation problems. Therefore, a semantic segmentation dataset of lotus seedpods was created, where a copy-and-paste data augmentation tool was employed to eliminate the class-imbalanced problem and improve model generalization ability. Afterwards, an improved YOLOv8-seg model was proposed to detect and segment the maturity of lotus seedpods. In the model, the convolutional block attention module (CBAM) was embedded in the neck network to extract distinguished features of different maturity stages with negligible computation cost. Wise-Intersection over Union (WIoU) regression loss function was adopted to refine the regression inference bias and improve the bounding box prediction accuracy. The experimental results showed that the proposed YOLOv8-seg model provides an effective method for “ripe” and “overripe” lotus seedpod detection and instance segmentation, where the mean average precision of segmentation mask (mAPmask) reaches 97.4% and 98.6%, respectively. In addition, the improved YOLOv8-seg exhibits high robustness and adaptability to complex illumination in a challenging environment. Comparative experiments were conducted using the proposed YOLOv8-seg and other state-of-the-art instance segmentation methods. The results showed that the improved model is superior to the Mask R-CNN and YOLACT models, with recall, precision, mAPbox and mAPmask being 96.5%, 94.3%, 97.8%, and 98%, respectively. The average running time and weight size of the proposed model are 25.9 ms and 7.4 M, respectively. The proposed model obtained the highest mAP for lotus seedpod maturity detection and segmentation while maintaining an appropriate model size and speed. Furthermore, based on the obtained segmentation model, 3D visualization of the lotus pond scene is performed, and cloud point of lotus seedpods is generated, which provides a theoretical foundation for robot harvesting in the lotus pond.

This study utilizes the Breast Ultrasound Image (BUSI) dataset to present a deep learning technique for breast tumor segmentation based on a modified UNet architecture. To improve segmentation accuracy, the model integrates attention mechanisms, such as the Convolutional Block Attention Module (CBAM) and Non-Local Attention, with advanced encoder architectures, including ResNet, DenseNet, and EfficientNet. These attention mechanisms enable the model to focus more effectively on relevant tumor areas, resulting in significant performance improvements. Models incorporating attention mechanisms outperformed those without, as reflected in superior evaluation metrics. The effects of Dice Loss and Binary Cross-Entropy (BCE) Loss on the model’s performance were also analyzed. Dice Loss maximized the overlap between predicted and actual segmentation masks, leading to more precise boundary delineation, while BCE Loss achieved higher recall, improving the detection of tumor areas. Grad-CAM visualizations further demonstrated that attention-based models enhanced interpretability by accurately highlighting tumor areas. The findings denote that combining advanced encoder architectures, attention mechanisms, and the UNet framework can yield more reliable and accurate breast tumor segmentation. Future research will explore the use of multi-modal imaging, real-time deployment for clinical applications, and more advanced attention mechanisms to further improve segmentation performance.

Heartbeat characteristic points are the main features of electrocardiogram (ECG), which can provide important information for ECG-based cardiac diagnosis. In this manuscript, we proposed a modified DenseNet to detect ECG characteristic points, including the onset, peak and termination points of P-wave, QRS complex wave and T-wave. The proposed DenseNet embedded the convolutional block attention module (CBAM) and feature pyramids in convolutional blocks. The model in this manuscript was trained using PhysioNet’s QT database, a dataset containing 105 2-lead dynamic recordings at 250 Hz. The mean± variance of the detection errors measured by our proposed method for the detection of the onset, peak and termination points of P-wave, the onset and termination points of QRS-wave, and the peak and termination points of T-wave were - 0.32 ± 18.08 ms, -0.56 ± 17.6 ms, -5.96 ± 16.84 ms, -5.8 ± 14.12 ms, -6.24 ± 18.76 ms, -0.2 ± 31.36 ms, 0.84 ± 27.24 ms, respectively. The results show that the deep learning network constructed in this manuscript can accurately detect the characteristic points of heartbeat, laying the foundation for automatic extraction of diagnosis-related key information from ECG.

Attention mechanism can focus on important features and suppress unnecessary features, to increase the representational power of the network, which plays an important role in visual tasks such as semantic segmentation. To this end, the UNET of Convolutional Block Attention Module (CBAM) is applied in semantic segmentation tasks to improve the performance of convolutional neural networks. The channel attention adopts maximum pooling and average pooling, and for the spatial attention, a smaller convolution kernel is proposed to reduce computation and loss of important features. Through experiments, the introduction of CBAM has a improvement in semantic segmentation tasks increasing the Validation Dice from 0.98 to 0.9871 in the Kaggle Carvana image segmentation dataset.

As a primary medium for emotional expression, human facial expressions carry rich informational value. Recent advancements in residual networks and attention mechanisms have broadened their application in expression classification, yet challenges persist in suboptimal key feature extraction and complex model training. To address these issues, this study proposes a novel facial expression recognition method integrating residual networks with attention mechanisms. The framework employs ResNet50 as the backbone network for feature extraction, enhanced by the Convolutional Block Attention Module (CBAM) to autonomously learn and prioritize critical features. Further innovations include reconstructing residual modules within the backbone network to optimize feature extraction and introducing a CAM-adjusted CBAM-ERF mechanism to mitigate neuronal suppression in specific regions, thereby accelerating network convergence and classification efficiency. Experimental results demonstrate the proposed residual network achieves 73.45% and 96.97% accuracy on the FER2013 and CK+ datasets, respectively.