A Symmetric Efficient Spatial and Channel Attention (ESCA) Module Based on Convolutional Neural Networks

As the core technologies represented by Internet of Things (IoT) technology and computer vision technology booming, object detection algorithms based on deep learning have received high attention in various fields. Object detection is not only a core problem in the field of computer vision, but also a prerequisite and basis for many computer vision tasks, and has important applications in the fields of autonomous driving and video surveillance. The task of object detection is to find all the targets of interest in an image and determine their category and location. Object detection includes two tasks: object classification and object localization. However, object classification focuses on the most discriminative part of the feature map, and object localization requires a feature map focusing on the whole region of the object. Therefore, in order to improve the detection accuracy, we propose a CBAM-RetinaNet network model.CBAM (Convolutional Block Attention Module) is a lightweight convolutional attention module, which combines channel and spatial attention mechanism modules. The channel attention module focuses on the meaningful information in the input image, which would lead to an increase in the accuracy of object classification. The spatial attention module focuses on the location information of the object, which may improve the accuracy of object localization. The experiment results show that the proposed achieves better object detection performance on the PASCAL VOC2007 dataset, and the accuracy is 2% higher than that of the RetinaNet model, which has good experimental results.

Impurity monitoring study for corn kernel harvesting based on machine vision and CPU-Net

Convolution neural networks (CNNs) have been achieving increasing attention for the artificial bandwidth extension (ABE) task recently. However, these methods use the flipped low-frequency phase to reconstruct speech signals, which may lead to the well-known invalid short-time Fourier Transform (STFT) problem. The convolutional operations only enable networks to construct informative features by fusing both channel-wise and spatial information within local receptive fields at each layer. In this paper, we introduce a Time-Frequency Network (TFNet) with channel attention (CA) and non-local (NL) modules for ABE. The TFNet exploits the information from both time and frequency domain branches concurrently to avoid the invalid STFT problem. To capture the channels and space dependencies, we incorporate the CA and NL modules to construct a proposed fully convolutional neural network for the time and frequency branches of TFNet. Experimental results demonstrate that the proposed method outperforms the competing method.

MRI image classification of brain tumors is critical for accurate and early diagnosis. New developments in deep learning have revealed that inserting attention mechanisms into convolutional neural networks can greatly improve classification performance. This work assesses the effectiveness of Squeeze-and-Excitation (SE) and Convolutional Block Attention Module (CBAM) integrated with the ResNet50 model for brain tumor classification. The SE module recalibrates channel-wise feature responses, while CBAM enhances feature representation by combining channel and spatial attention. The SE and CBAM are combined with ResNet50, greatly increasing classification accuracy, precision, and recall when compared to the basic model, according to experimental results on MRI datasets. The suggested model employs attention mechanisms to focus valuable information selectively and suppress irrelevant information. The experiments on publicly available brain tumor datasets display great improvements over basic CNN models, with precision, recall, accuracy, as well as F1 score at 0.9914, 0.9903, 0.9945, and 0.9908, respectively. From these results, the importance of attention mechanisms in deep learning models for medical imaging is highlighted, which suggests that SE and CBAM modules can be available as more dependable and effective instruments for brain tumor classification in clinical settings. Future studies should investigate transformer-based and hybrid attention techniques to enhance automated brain tumor categorization.

Sheep behavior recognition helps to monitor the health status of sheep and prevent the outbreak of infectious diseases. Aiming at the problems of low detection accuracy and slow speed due to the crowding of sheep in real farming scenarios, which can easily obscure each other, this study proposes a lightweight sheep behavior recognition model based on the YOLOv8n model. First, the Convolutional Block Attention Module (CBAM) is introduced and improved in the YOLOv8n model, and the channel attention module and spatial attention module are changed from serial to parallel to construct a novel attention mechanism, PCBAM, to enhance the network’s attention to the sheep and eliminate redundant background information; second, the ordinary convolution in the backbone network is replaced with depth-separable convolution, which effectively reduces the number of parameters in the model and reduces the computational complexity. The study takes the housed breeding sheep as the test object, installs a camera diagonally above the sheep pen to collect images and makes a data set for testing, and in order to verify the superiority of the PD-YOLO model, compares it with a variety of target detection models. The experimental results show that the mean average precision (mAP) of the model proposed in this paper are 95.8%, 98.9%, and 96.2% for the three postures of sheep lying, feeding, and standing, respectively, which are 8.5%, 0.8%, and 0.8% higher than those of YOLOv8n, respectively, and the size of the model has been reduced by 13.3% and the amount of computation has been reduced by 12.1%. The inference speed reaches 52.1 FPS per second, which is better than other models in meeting the real-time detection requirement. To verify the practicality of this research method, the PD-YOLO model was deployed on the RK3399Pro development board for testing, and a high inference speed was achieved. It can provide effective technical support for sheep smart farming.

Lane detection based on semantic segmentation can achieve high accuracy, but, in recent years, it does not have a mobile-friendly cost, which is caused by the complex iteration and costly convolutions in convolutional neural networks (CNNs) and state-of-the-art (SOTA) models based on CNNs, such as spatial CNNs (SCNNs). Although the SCNN has shown its capacity to capture the spatial relationships of pixels across rows and columns of an image, the computational cost and memory requirement needed cannot be afforded with mobile lane detection. Inspired by the channel attention and self-attention machine, we propose an integrated coordinate attention (ICA) module to capture the spatial-wise relationships of pixels. Furthermore, due to the lack of enhancement in the channel dimension, we created an efficient network with a channel-enhanced coordinate attention block named CCA, composed of ICA and other channel attention modules, for all-dimension feature enhancement. As a result, by replacing many repeated or iterative convolutions with the attention mechanism, CCA reduces the computational complexity. Thus, our method achieves a balance of accuracy and speed and has better performance on two lane datasets—TuSimple and ILane. At less than a few tenths of the computational cost, our CCA achieves superior accuracy compared to the SCNN. These results show that the low cost and great performance of our design enable the use of the lane detection task in autopilot scenarios.

The primary focus of this research article is to advance convolutional neural networks (CNNs) through the incorporation of Squeeze and Excitation (SE) and Convolutional Block Attention Module (CBAM) blocks, with an emphasis on recalibrating channel wise features. This approach highlights channel relationships, with the SE block seamlessly integrated into a pre-trained DenseNet CNN model to recalibrate channel wise feature responses by explicitly modeling channel interdependencies. The study demonstrates the construction of SENet and CBAM architectures with SE and CBAM blocks on pretrained DenseNet models using transfer learning, achieving exceptional generalization on a cataract image dataset. The proposed architectures utilize layers of compressed network blocks that can establish direct connections with preceding layers through adjustable weight modifications. Comprehensive evaluations indicate that the DenseSE and DenseCBAM models both achieve 99% testing accuracy and exhibit strong performance across various metrics. Notably, DenseNet169 with SE and CBAM blocks attained 99% precision, surpassing the baseline model’s 95% precision. Furthermore, the Grad CAM technique enhances cataract disease identification, with the DenseCBAM and DenseSE models generating detailed heatmaps that support healthcare professionals in making swift and accurate retinal diagnoses.

In recent years, vein biometrics has gained significant attention due to its high security and privacy features. While deep neural networks have become the predominant classification approaches for their ability to automatically extract discriminative vein features, they still face certain drawbacks: 1) Existing transformer-based vein classifiers struggle to capture interactive information among different attention modules, limiting their feature representation capacity; 2) Current label enhancement methods, although effective in learning label distributions for classifier training, fail to model long-range relations between classes. To address these issues, we present ALE-IVT, an Attention Label Enhancement-based Interactive Vein Transformer for palm-vein recognition. First, to extract vein features, we propose an interactive vein transformer (IVT) consisting of three branches, namely spatial attention, channel attention, and convolutional module. In order to enhance performance, we integrate an interactive module that facilitates the sharing of discriminative features among the three branches. Second, we explore an attention-based label enhancement (ALE) approach to learn label distribution. ALE employs a self-attention mechanism to capture correlation between classes, enabling the inference of label distribution for classifier training. As self-attention can model long-range dependencies between classes, the resulting label distribution provides enhanced supervised information for training the vein classifier. Finally, we combine ALE with IVT to create ALE-IVT, trained in an end-to-end manner to boost the recognition accuracy of the IVT classifier. Our experiments on three public datasets demonstrate that our IVT model surpasses existing state-of-the-art vein classifiers. In addition, ALE outperforms current label enhancement approaches in term of recognition accuracy.

In recent years, the research of deep learning has received extensive attention, and many breakthroughs have been made in various fields. On this basis, a neural network with the attention mechanism has become a research hotspot. In this paper, we try to solve the image classification task by implementing channel and spatial attention mechanism which improve the expression ability of neural network model. Different from previous studies, we propose an attention module consisting of channel attention module (CAM) and spatial attention module (SAM). The proposed module derives attention graphs from channel dimension and spatial dimension respectively, then the input features are selectively learned according to the importance of the features. Besides, this module is lightweight and can be easily integrated into image classification algorithms. In the experiment, we combine the deep residual network model with the attention module and the experimental results show that the proposed method brings higher image classification accuracy. The channel attention module adds weight to the signals on different convolution channels to represent the correlation. For different channels, the higher the weight, the higher the correlation which required more attention. The main function of spatial attention is to capture the most informative part in the local feature graph, which is a supplement to channel attention. We evaluate our proposed module based on the ImageNet-1K and Cifar-100 respectively. Through a large number of comparative experiments, our proposed model achieved outstanding performance.

Brain-computer interfaces (BCI) allow for the brain to communicate with electronic devices. Concerning the BCI paradigms, motor imagery uses brain signals to decode an imagined movement. However, this is a hard task given the low signal-to-noise ratio. Usually, the main steps in BCI models are pre-processing, feature extraction, and classification. In recent years, Convolutional Neural Networks (CNNs) have been gaining relevance in several areas of science due to their feature extraction, translation invariance, and parameter sharing capa-bilities. Another, more recent way of feature extraction is using attention mechanisms, which are layers of neural networks based on human attention and have the ability to highlight important features. A variation of the attention mechanism is the Convolutional Block Attention Module, which combines the CNN structure with the attention mechanism. In this work, we propose a new model that joins the core architecture of EEGNet, a compact CNN widely used in the literature, with the Convolutional Block Attention Module and residual connections. The residual connections were introduced to lower data degradation throughout the model. The results highlight the residual connection's importance for the performance of the model. The proposed model obtained a kappa result 5.2% better than the EEGNet with a p-value less than 0.01 on BCI Competition IV dataset 2a, which is a well-known dataset for Motor Imagery. Furthermore, the proposal was better than EEGNet for most subjects and had the best-worst case.

IntroductionThe cigar leaves moisture content (CLMC) is a critical parameter for controlling curing barn conditions. Along with the continuous advancement of deep learning (DL) technologies, convolutional neural networks (CNN) have provided a way of thinking for the non-destructive estimation of CLMC during the air-curing process. Nevertheless, relying merely on single-perspective imaging makes it difficult to comprehensively capture the complementary morphological features of the front and back sides of cigar leaves during the air-curing process.MethodsThis study constructed a dual-view image dataset covering the air-curing process, and proposes a regression framework named CADFFNet (channel attention weight-based dual-branch feature fusion network) for the non-destructive estimation of CLMC during the curing process based on dual-view RGB images. Firstly, the model utilizes two independent and parallel ResNet as its backbone structure to capture the heterogeneous features of dual-view images. Secondly, the Dual Efficient Channel Attention (DECA) module is introduced to dynamically adjust the channel attention weights of the features, thereby facilitating interaction between the two branches. Lastly, a Multi-scale convolutional feature fusion (MSCFF) module is designed for the deep fusion of features from the front and back images to aggregate multi-scale features for robust regression.ResultsOn five-fold cross-validation, CADFFNet attains R2 of 0.974±0.007 and mean absolute error (MAE) of 3.80±0.37%. On an independent cross-region, cross-variety testing set, it maintains strong generalization (R2=0.899, MAE=5.82%), compared with the classic CNN models ResNet18, GoogLeNet, VGG19Net, DenseNet121, and MobileNetV2, its R2 value has increased by 0.047, 0.041, 0.055, 0.098, and 0.090 respectively.DiscussionGenerally, the proposed CADFFNet offers an efficient and convenient method for non-destructive detection of CLMC, providing a theoretical basis for automating the air-curing process. It also provides a new perspective for moisture content prediction during the drying process of other crops, such as tea, asparagus, and mushrooms.

Accurate and timely detection of kidney stones is crucial for effective medical intervention and treatment planning. However, existing detection methods often struggle with challenges related to sensitivity, precision, and the ability to process complex and variable medical images. In this study, an advanced kidney stone detection system is developed using the latest object detection algorithm, You Only Look Once version 10 (YOLOv10), integrated with channel attention mechanisms to enhance model performance. This combination aims to improve detection accuracy by enabling the network to focus more precisely on critical regions in medical images, particularly in Computed Tomography (CT) scans, where kidney stones may appear in varying shapes, sizes, and intensities. The proposed system begins with data augmentation techniques, such as rotation, scaling, and contrast adjustments, to enhance the model’s generalization ability across different image conditions and patient profiles. YOLOv10 was selected due to its lightweight architecture, high detection speed, and enhanced performance in small object detection tasks. To further improve feature extraction, channel attention mechanisms such as Squeeze-and-Excitation (SE) blocks or Efficient Channel Attention (ECA) modules are incorporated. These modules enable the network to selectively focus on the most informative feature channels associated with kidney stone regions, while suppressing irrelevant background information, thereby improving the distinction between stones and surrounding tissues. The model is trained and fine-tuned using a diverse CT scan dataset containing various types and sizes of kidney stones. Evaluation results demonstrate that the proposed model achieves a high detection accuracy of 93.7% with a very low loss of 0.18. It exhibits stability without issues like overfitting, underfitting, or local minima entrapment, making it a highly reliable tool for clinical applications.

The integration of Multi-layer Perceptrons (MLPs), Convolutional Neural Networks (CNNs), and attention mechanisms has been demonstrated to significantly enhance model performance across various computer vision tasks. In this paper, a novel lightweight neural network architecture, FasterMLP, is proposed to achieve high computational efficiency and accuracy, particularly in resource-constrained and real-time applications. FasterMLP is designed to combine the local connectivity and weight-sharing properties of CNNs with the global feature representation capabilities of MLPs, while feature extraction is enhanced through the Convolutional Block Attention Module and spatial dimensions are effectively reduced using Haar wavelet downsampling without sacrificing critical feature information. The architecture, structured into four stages, has been rigorously evaluated on multiple benchmarks. On the ImageNet-1K dataset, a top-1 accuracy 3.9% higher than that of MobileViT-XXS is achieved by FasterMLP-S, while being 2 and 2.7 faster on GPU and CPU, respectively. On the COCO dataset, the performance of FasterMLP-L is shown to be comparable to FasterNet-L with significantly fewer parameters, and on the Cityscapes dataset, a mean Intersection-over-Union of 81.7% is achieved, surpassing existing methods such as CCNet and DANet. These results demonstrate that FasterMLP can effectively balance computational efficiency and accuracy, making it particularly suitable for visual perception tasks in resource-constrained and real-time environments such as autonomous driving. Code is available at https://github.com/windisl/FasterMLP.

Medical image analysis is highly susceptible to noise during acquisition and transmission, significantly degrading diagnostic accuracy. This study proposes a robust Attention-Based Squeeze-Excitation Residual Autoencoder (ASE-RAE) for denoising medical images corrupted by Gaussian and Salt-and-Pepper (SP) noise. The network integrates residual skip connections in both encoder and decoder paths, combined with Squeeze-and-Excitation (SE) blocks and a Convolutional Block Attention Module (CBAM) to improve channel and spatial feature representation. The model is benchmarked against traditional denoising techniques, including Non-Local Means Filter (NLMF), Median Filter (MF), Wavelet Transform (WT), Gaussian Filter (GF), and Convolutional Neural Networks (CNNs). Quantitative evaluation using Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), and Mean Squared Error (MSE) demonstrates that the proposed method consistently outperforms all baseline models across varying noise intensities (5%–75%) on the IQ-OTH/NCCD lung CT dataset. Furthermore, ablation studies confirm the contribution of each module (SE and CBAM) to performance gains. The proposed architecture preserves diagnostic features while suppressing both random and structured noise, and its convolutional SE implementation reduces computational overhead compared to conventional fully connected attention blocks. This work highlights a scalable, interpretable solution for image denoising in clinical workflows and sets a foundation for real-time deployment in AI-assisted diagnostics.

Frequency-based CNN and attention module for acoustic scene classification