Attention graph: Learning effective visual features for large-scale image classification

In recent years, attention mechanisms have shown great potential in various computer vision tasks. However, most existing methods focus on developing more complex attention modules for better performance, which inevitably increases the complexity of the model. To overcome performance and complexity tradeoffs, this paper proposes efficient spatial and channel attention (ESCA), a symmetric, comprehensive, and efficient attention module. By analyzing squeeze-and-excitation (SE), convolutional block attention module (CBAM), coordinate attention (CA), and efficient channel attention (ECA) modules, we abandon the dimension-reduction operation of SE module, verify the negative impact of global max pooling (GMP) on the model, and apply a local cross-channel interaction strategy without dimension reduction to learn attention. We not only care about the channel features of the image, we also care about the spatial location of the target on the image, and we take into account the effectiveness of channel attention, so we designed the symmetric ESCA module. The ESCA module is effective, as demonstrated by its application in the ResNet-50 classification benchmark. With 26.26 M parameters and 8.545 G FLOPs, it introduces a mere 0.14% increment in FLOPs while achieving over 6.33% improvement in Top-1 accuracy and exceeding 3.25% gain in Top-5 accuracy. We perform image classification and object detection tasks on ResNet, MobileNet, YOLO, and other architectures on popular datasets such as Mini ImageNet, CIFAR-10, and VOC 2007. Experiments show that ESCA can achieve great improvement in model accuracy at a very small cost, and it performs well among similar models.

In this paper, we focus on the semantic image synthesis task that aims at transferring semantic label maps to photo-realistic images. Existing methods lack effective semantic constraints to preserve the semantic information and ignore the structural correlations in both spatial and channel dimensions, leading to unsatisfactory blurry and artifact-prone results. To address these limitations, we propose a novel Dual Attention GAN (DAGAN) to synthesize photo-realistic and semantically-consistent images with fine details from the input layouts without imposing extra training overhead or modifying the network architectures of existing methods. We also propose two novel modules, i.e., position-wise Spatial Attention Module (SAM) and scale-wise Channel Attention Module (CAM), to capture semantic structure attention in spatial and channel dimensions, respectively. Specifically, SAM selectively correlates the pixels at each position by a spatial attention map, leading to pixels with the same semantic label being related to each other regardless of their spatial distances. Meanwhile, CAM selectively emphasizes the scale-wise features at each channel by a channel attention map, which integrates associated features among all channel maps regardless of their scales. We finally sum the outputs of SAM and CAM to further improve feature representation. Extensive experiments on four challenging datasets show that DAGAN achieves remarkably better results than state-of-the-art methods, while using fewer model parameters.

Case instance segmentation of small farmland based on Mask R-CNN of feature pyramid network with double attention mechanism in high resolution satellite images

It aimed to analyze the value of deep learning algorithm combined with magnetic resonance imaging (MRI) in the risk diagnosis and prognosis of endometrial cancer (EC). Based on the deep learning convolutional neural network (CNN) architecture residual network with 101 layers (ResNet-101), spatial attention and channel attention modules were introduced to optimize the model. A retrospective collection of MRI image data from 210 EC patients was used for model segmentation and reconstruction, with 140 cases as the test set and 70 cases as the validation set. The performance was compared with traditional ResNet-101 model, ResNet-101 model based on spatial attention mechanism (SA-ResNet-101), and ResNet-101 model based on channel attention mechanism (CA-ResNet-101), using accuracy (AC), precision (PR), recall (RE), and F1 score as evaluation metrics. Among the 70 cases in the validation set, there were 45 cases of low-risk EC and 25 cases of high-risk EC. Using ROC curve analysis, it was found that the area under the curve (AUC) for the diagnosis of high-risk EC of the proposed model in this article (0.918) was visibly larger as against traditional ResNet-101 (0.613), SA-ResNet-101 (0.760), and CA-ResNet-101 models (0.758). The AC, PR, RE, and F1 values of the proposed model for the diagnosis of EC risk were visibly higher (P < 0.05). In the validation set, postoperative recurrence occurred in 13 cases and did not occur in 57 cases. Using ROC curve analysis, it was found that the AUC for postoperative recurrence prediction of the patients by the proposed model (0.926) was visibly larger as against traditional ResNet-101 (0.620), SA-ResNet-101 (0.729), and CA-ResNet-101 models (0.767). The AC, PR, RE, and F1 values of the proposed model for postoperative recurrence prediction were visibly higher (P < 0.05). The proposed model in this article, assisted by MRI, presented superior performance in diagnosing high-risk EC patients, with higher sensitivity (Sen) and specificity (Spe), and also demonstrated excellent predictive AC in postoperative recurrence prediction.

In recent years, vein biometrics has gained significant attention due to its high security and privacy features. While deep neural networks have become the predominant classification approaches for their ability to automatically extract discriminative vein features, they still face certain drawbacks: 1) Existing transformer-based vein classifiers struggle to capture interactive information among different attention modules, limiting their feature representation capacity; 2) Current label enhancement methods, although effective in learning label distributions for classifier training, fail to model long-range relations between classes. To address these issues, we present ALE-IVT, an Attention Label Enhancement-based Interactive Vein Transformer for palm-vein recognition. First, to extract vein features, we propose an interactive vein transformer (IVT) consisting of three branches, namely spatial attention, channel attention, and convolutional module. In order to enhance performance, we integrate an interactive module that facilitates the sharing of discriminative features among the three branches. Second, we explore an attention-based label enhancement (ALE) approach to learn label distribution. ALE employs a self-attention mechanism to capture correlation between classes, enabling the inference of label distribution for classifier training. As self-attention can model long-range dependencies between classes, the resulting label distribution provides enhanced supervised information for training the vein classifier. Finally, we combine ALE with IVT to create ALE-IVT, trained in an end-to-end manner to boost the recognition accuracy of the IVT classifier. Our experiments on three public datasets demonstrate that our IVT model surpasses existing state-of-the-art vein classifiers. In addition, ALE outperforms current label enhancement approaches in term of recognition accuracy.

In this paper, we propose a novel fusion framework based on a dense network with channel and spatial attention (CSpA-DN) for PET and MR images. In our approach, an encoder composed of the densely connected neural network is constructed to extract features from source images, and a decoder network is leveraged to yield the fused image from these features. Simultaneously, a self-attention mechanism is introduced in the encoder and decoder to further integrate local features along with their global dependencies adaptively. The extracted feature of each spatial position is synthesized by a weighted summation of those features at the same row and column with this position via a spatial attention module. Meanwhile, the interdependent relationship of all feature maps is integrated by a channel attention module. The summation of the outputs of these two attention modules is fed into the decoder and the fused image is generated. Experimental results illustrate the superiorities of our proposed CSpA-DN model compared with state-of-the-art methods in PET and MR images fusion according to both visual perception and objective assessment.

The refined segmentation of nuclei and the cytoplasm is the most challenging task in the automation of cervical cell screening. The U-Shape network structure has demonstrated great superiority in the field of biomedical imaging. However, the classical U-Net network cannot effectively utilize mixed domain information and contextual information, and fails to achieve satisfactory results in this task. To address the above problems, a module based on global dependency and local attention (GDLA) for contextual information modeling and features refinement, is proposed in this study. It consists of three components computed in parallel, which are the global dependency module, the spatial attention module, and the channel attention module. The global dependency module models global contextual information to capture a priori knowledge of cervical cells, such as the positional dependence of the nuclei and cytoplasm, and the closure and uniqueness of the nuclei. The spatial attention module combines contextual information to extract cell boundary information and refine target boundaries. The channel and spatial attention modules are used to provide adaption of the input information, and make it easy to identify subtle but dominant differences of similar objects. Comparative and ablation experiments are conducted on the Herlev dataset, and the experimental results demonstrate the effectiveness of the proposed method, which surpasses the most popular existing channel attention, hybrid attention, and context networks in terms of the nuclei and cytoplasm segmentation metrics, achieving better segmentation performance than most previous advanced methods.

Sheep behavior recognition helps to monitor the health status of sheep and prevent the outbreak of infectious diseases. Aiming at the problems of low detection accuracy and slow speed due to the crowding of sheep in real farming scenarios, which can easily obscure each other, this study proposes a lightweight sheep behavior recognition model based on the YOLOv8n model. First, the Convolutional Block Attention Module (CBAM) is introduced and improved in the YOLOv8n model, and the channel attention module and spatial attention module are changed from serial to parallel to construct a novel attention mechanism, PCBAM, to enhance the network’s attention to the sheep and eliminate redundant background information; second, the ordinary convolution in the backbone network is replaced with depth-separable convolution, which effectively reduces the number of parameters in the model and reduces the computational complexity. The study takes the housed breeding sheep as the test object, installs a camera diagonally above the sheep pen to collect images and makes a data set for testing, and in order to verify the superiority of the PD-YOLO model, compares it with a variety of target detection models. The experimental results show that the mean average precision (mAP) of the model proposed in this paper are 95.8%, 98.9%, and 96.2% for the three postures of sheep lying, feeding, and standing, respectively, which are 8.5%, 0.8%, and 0.8% higher than those of YOLOv8n, respectively, and the size of the model has been reduced by 13.3% and the amount of computation has been reduced by 12.1%. The inference speed reaches 52.1 FPS per second, which is better than other models in meeting the real-time detection requirement. To verify the practicality of this research method, the PD-YOLO model was deployed on the RK3399Pro development board for testing, and a high inference speed was achieved. It can provide effective technical support for sheep smart farming.

This paper presents a dual-structured convolutional neural network (CNN) for image classification, which integrates two parallel branches: CNN-A with spatial attention and CNN-B with channel attention. The spatial attention module in CNN-A dynamically emphasizes discriminative regions by aggregating channel-wise information, while the channel attention mechanism in CNN-B adaptively recalibrates feature channel importance. The extracted features from both branches are fused through concatenation, enhancing the model’s representational capacity by capturing complementary spatial and channel-wise dependencies. Extensive experiments on a 12-class image dataset demonstrate the superiority of the proposed model over state-of-the-art methods, achieving 98.06% accuracy, 96.00% precision, and 98.01% F1-score. Despite a marginally longer training time, the model exhibits robust convergence and generalization, as evidenced by stable loss curves and high per-class recognition rates (&gt;90%). The results validate the efficacy of dual attention mechanisms in improving feature discrimination for complex image classification tasks.

Electricity is the basis for everyday life, and defect detection of insulators of high-voltage transmission lines is the key to ensuring power transmission. In order to overcome the problems of low accuracy of traditional target detection algorithms for small targets, weak representation ability of feature maps and little key information extraction, an improved CBAM attention-based insulator defect detection method CBAM-YOLOv8 based on YOLOv8 was proposed. The core is to apply the combination of Channel Attention Module and Spatial Attention Module to process the channel attention and spatial attention modules respectively for the input feature layers. Experiments show that the AP in the CPLID dataset is improved by 4.7% and the FPS is reduced by 2.7% compared with the original YOLOv8, which proves that the proposed method can maintain a high detection speed while greatly improving the detection accuracy, providing a more effective and safer solution for the detection of high-voltage transmission lines, and greatly reducing the labor cost and operation risk.

Lung disease diagnosis remains a significant clinical challenge due to the similarity in radiological features across various conditions such as COPD, pneumonia, tuberculosis, COVID‐19, and lung cancer. Manual interpretation of chest CT scans is time‐consuming and subject to inter‐observer variability, particularly in resource‐limited settings. To address these challenges, this study proposes a novel deep learning framework Multiscale Convolutional Neural Networks with Attention Mechanism (MCNN‐AM) for automated classification of lung diseases into six categories, including normal lungs. The model leverages multiscale convolutional layers to extract both localized and global features, enabling better discrimination between diseases with overlapping characteristics. A dynamic attention mechanism, comprising both spatial and channel attention modules, is integrated to emphasize disease‐relevant regions and suppress background noise, enhancing the model's diagnostic focus. Additionally, depthwise separable convolutions are utilized to reduce computational complexity while preserving feature richness. The MCNN‐AM model is trained and evaluated on publicly available datasets, comprising 6000 training images and 1200 testing images equally distributed across all classes. The model achieves a classification accuracy of 96.84%, outperforming state‐of‐the‐art models such as ResNet50, DenseNet121, and InceptionV3 in terms of precision, recall, F1‐score, sensitivity, and specificity. Ablation studies further validate the critical role of the attention modules in achieving high performance. These results demonstrate the potential of MCNN‐AM as a reliable, scalable tool for computer‐aided diagnosis of lung diseases.

After decades of extensive research, the field of 2D face recognition is fruitful. However, 2D face recognition is sensitive to changes in pose, facial expression and illumination. 3D face recognition has great potential due to the inherent invariance of pose and illumination changes, but these 3D face images are still difficult to acquire due to several issues such as cost and accessibility. In contrast, RGB-D images captured by low-cost depth sensors such as Kinect are relatively easy to acquire, and research on such low-quality inputs is still limited. In this thesis, an end-to-end multimodal fusion method based on spatial and channel attention is proposed to effectively fuse two image modalities, RGB and depth, to enhance RGB-D face recognition. Our work is specified as follows: firstly, three branches combined with the spatial and channel attention modules are established on the basis of ResNet18, so that the features of three modalities (RGB, depth map, and their fusion modalities, respectively) are obtained. The features under these three modalities are then fused, fed into a shared layer, and the fused features are further learned for deeper discriminative features and passed through a spatial attention vectorization module at the end. The method achieves good results on the Intellifusion RGBD dataset, while we perform ablation trials, which well reveal the contribution of each component of our method to the final performance.

FBSA-Net: A novel model based on attention mechanisms for emotion recognition in VR and 2D scenes

The development of high-resolution remote sensing and deep learning techniques has significantly advanced the extraction of urban structures. However, densely built areas pose challenges due to the complex geometry and spatial distribution of structures, resulting in uncertainties in their frequency-domain characterization. To resolve these issues, this research proposes WaveFuseNet, an effective network designed for multiscale building extraction sensitive to frequency information. Within its encoder, WaveFuseNet incorporates a Multiscale Wavelet Transform Convolutional (MWTC) module to extract both high-frequency and low-frequency features across scales. Furthermore, the Adaptive Spatial and Channel Attention (ASCA) module is integrated to enhance global context modeling by strengthening the synergy between spatial and channel attention. The decoder employs an Adaptive Feature Fusion Decoder that performs learnable, weighted fusion of multiscale features and upsampling operations to preserve spatial-frequency characteristics. Comprehensive evaluations conducted on the WHU dataset, the Dataset of Building Instances of Typical Cities in China, and the Massachusetts Buildings Dataset demonstrate the model’s excellent performance, reaching IoU values of 91.15%, 75.76%, and 72.56%, respectively, with only 16.13 M parameters. These results demonstrate that WaveFuseNet is an effective and efficient scheme for extracting urban buildings at large scales, providing valuable support for smart city development and sustainable urban planning. Access to the source code is provided through https://github.com/786942053/WavefuseNet/blob/main/README.md.

Attention mechanisms are rapidly being noticed in the computer vision community. However, existing works focus on the design of a single attention module, and then employ the same attention module in all attention layers of an entire network, rendering sub-optimal performance. In this paper, we address a learning-to-attention problem by proposing Switchable Attention (SA), which learns to select different kinds of attention modules for variant blocks of a Deep Neural Network (DNN). SA employs three distinct scopes to compute the attention map, including local spatial attention (LSA), global spatial attention (GSA) and channel attention (CA). Especially, the introduced trainable parameters can be optimized with the network in an end-to-end manner. SA is a light-weight module which can be embedded in existing networks with little overhead of computation. We conduct several quantitative experiments, and SA boosts the performance of the baseline on various challenging benchmarks, such as CIFAR-100, ImageNet-1K, and MS COCO and different computer vision tasks, such as Image Classification and Object detection. Notably, SA outperforms the most attention methods with ResNet-50 on ImageNet-1K.