Optimization and Practice of Convolutional Neural Networks in Image Classification Tasks

MRI image classification of brain tumors is critical for accurate and early diagnosis. New developments in deep learning have revealed that inserting attention mechanisms into convolutional neural networks can greatly improve classification performance. This work assesses the effectiveness of Squeeze-and-Excitation (SE) and Convolutional Block Attention Module (CBAM) integrated with the ResNet50 model for brain tumor classification. The SE module recalibrates channel-wise feature responses, while CBAM enhances feature representation by combining channel and spatial attention. The SE and CBAM are combined with ResNet50, greatly increasing classification accuracy, precision, and recall when compared to the basic model, according to experimental results on MRI datasets. The suggested model employs attention mechanisms to focus valuable information selectively and suppress irrelevant information. The experiments on publicly available brain tumor datasets display great improvements over basic CNN models, with precision, recall, accuracy, as well as F1 score at 0.9914, 0.9903, 0.9945, and 0.9908, respectively. From these results, the importance of attention mechanisms in deep learning models for medical imaging is highlighted, which suggests that SE and CBAM modules can be available as more dependable and effective instruments for brain tumor classification in clinical settings. Future studies should investigate transformer-based and hybrid attention techniques to enhance automated brain tumor categorization.

Brain-computer interfaces (BCI) allow for the brain to communicate with electronic devices. Concerning the BCI paradigms, motor imagery uses brain signals to decode an imagined movement. However, this is a hard task given the low signal-to-noise ratio. Usually, the main steps in BCI models are pre-processing, feature extraction, and classification. In recent years, Convolutional Neural Networks (CNNs) have been gaining relevance in several areas of science due to their feature extraction, translation invariance, and parameter sharing capa-bilities. Another, more recent way of feature extraction is using attention mechanisms, which are layers of neural networks based on human attention and have the ability to highlight important features. A variation of the attention mechanism is the Convolutional Block Attention Module, which combines the CNN structure with the attention mechanism. In this work, we propose a new model that joins the core architecture of EEGNet, a compact CNN widely used in the literature, with the Convolutional Block Attention Module and residual connections. The residual connections were introduced to lower data degradation throughout the model. The results highlight the residual connection's importance for the performance of the model. The proposed model obtained a kappa result 5.2% better than the EEGNet with a p-value less than 0.01 on BCI Competition IV dataset 2a, which is a well-known dataset for Motor Imagery. Furthermore, the proposal was better than EEGNet for most subjects and had the best-worst case.

In recent years, attention mechanisms have shown great potential in various computer vision tasks. However, most existing methods focus on developing more complex attention modules for better performance, which inevitably increases the complexity of the model. To overcome performance and complexity tradeoffs, this paper proposes efficient spatial and channel attention (ESCA), a symmetric, comprehensive, and efficient attention module. By analyzing squeeze-and-excitation (SE), convolutional block attention module (CBAM), coordinate attention (CA), and efficient channel attention (ECA) modules, we abandon the dimension-reduction operation of SE module, verify the negative impact of global max pooling (GMP) on the model, and apply a local cross-channel interaction strategy without dimension reduction to learn attention. We not only care about the channel features of the image, we also care about the spatial location of the target on the image, and we take into account the effectiveness of channel attention, so we designed the symmetric ESCA module. The ESCA module is effective, as demonstrated by its application in the ResNet-50 classification benchmark. With 26.26 M parameters and 8.545 G FLOPs, it introduces a mere 0.14% increment in FLOPs while achieving over 6.33% improvement in Top-1 accuracy and exceeding 3.25% gain in Top-5 accuracy. We perform image classification and object detection tasks on ResNet, MobileNet, YOLO, and other architectures on popular datasets such as Mini ImageNet, CIFAR-10, and VOC 2007. Experiments show that ESCA can achieve great improvement in model accuracy at a very small cost, and it performs well among similar models.

Wind turbine rolling bearings are crucial components for ensuring the reliability and stability of wind power systems. Their failure can lead to significant economic losses and equipment downtime. Therefore, the accurate diagnosis of bearing faults is of great importance. Although existing deep learning fault diagnosis methods have achieved certain results, they still face limitations such as inadequate feature extraction capabilities, insufficient generalization to complex working conditions, and ineffective multi-scale feature capture. To address these issues, this paper proposes an advanced fault diagnosis method named the two-stream feature fusion convolutional neural network (TSFFResNet-Net). Firstly, the proposed method combines the advantages of one-dimensional convolutional neural networks (1D-ResNet) and two-dimensional convolutional neural networks (2D-ResNet). It transforms one-dimensional vibration signals into two-dimensional images through the empirical wavelet transform (EWT) method. Then, parallel convolutional kernels in 1D-ResNet and 2D-ResNet are used to extract multi-scale features, respectively. Next, the Convolutional Block Attention Module (CBAM) is introduced to enhance the network’s ability to capture key features by focusing on important features in specific channels or spatial areas. After feature fusion, CBAM is introduced again to further enhance the effect of feature fusion, ensuring that the features extracted by different network branches can be effectively integrated, ultimately providing more accurate input features for the classification task of the fully connected layer. The experimental results demonstrate that the proposed method outperforms other traditional methods and advanced convolutional neural network models on different datasets. Compared with convolutional neural network models such as LeNet-5, AlexNet, and ResNet, the proposed method achieves a significantly higher accuracy on the test set, with a stable accuracy of over 99%. Compared with other models, it shows better generalization and stability, effectively improving the overall performance of rolling bearing vibration signal fault diagnosis. The method provides an effective solution for the intelligent fault diagnosis of wind turbine rolling bearings.

Acquiring the growth information of tobacco plants quickly and accurately is an important prerequisite for their daily management. In view of the problem that the convolutional neural network is easily affected by background information, this study introduces the Convolutional Block Attention Module into the U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2-</sup> Net. This method is mainly composed of an encoder and a decoder. Before the feature map is decoded, it will go through the Convolutional Block Attention Module to filter more important features and inhibit the invalid features. In the decoder, in order to fuse the features of objects in different scales, the feature maps in the decoder are up-sampled and then concatenated. This information aggregation solves the problem of difficult object detection at different scales to some extent. The results of this method are compared with four current mainstream deep learning models(U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net, DeeplabV3+, PSP-Net, U-Net). The result shows: 1) The U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net network based on the Convolutional Block Attention Module has the highest extraction accuracy for the UAV visible light remote senseing of tobacco(accuracy was 98.53%, recall was 96.00%,F1-Score was 96.38%, kappa coefficient was 0.95, IOU was 93.02%); 2)In the extraction of the three test areas, the extraction value of this method is highly consistent with the label area value(the area ratios of a total of 13 plants in the three regions were kept between 90% and 100%),the effect is much higher than that of DeeplabV3+ and PSP-Net, and slightly higher than the extraction effect of U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net and U-Net. 3)In the segmentation of three typical canopy closure regions of A, B, and C, most of the accuracy factors in different regions can reach 94% and above. It can be seen that the method in this study has certain advantages in feature extraction and has certain value in practical application.

The development of Facial Expression Recognition (FER) technology has significantly enhanced the naturalness and intuitiveness of human-robot interaction. In the field of service robots, particularly in applications such as production assistance, caregiving, and daily service communication, efficient FER capabilities are crucial. However, existing Convolutional Neural Network (CNN) models still have limitations in terms of feature representation and recognition accuracy for facial expressions. To address these challenges, we propose CAGNet, a novel network that combines multiscale feature aggregation and attention mechanisms. CAGNet employs a deep learning-based hierarchical convolutional architecture, enhancing the extraction of features at multiple scales through stacked convolutional layers. The network integrates the Convolutional Block Attention Module (CBAM) and Global Average Pooling (GAP) modules to optimize the capture of both local and global features. Additionally, Batch Normalization (BN) layers and Dropout techniques are incorporated to improve model stability and generalization. CAGNet was evaluated on two standard datasets, FER2013 and CK+, and the experiment results demonstrate that the network achieves accuracies of 71.52% and 97.97%, respectively, in FER. These results not only validate the effectiveness and superiority of our approach but also provide a new technical solution for FER. Furthermore, CAGNet offers robust support for the intelligent upgrade of service robots.

Focusing on the problems of complex structure and low feature extraction efficiency that exist in some traditional neural network algorithms, an improved convolutional neural network, combined with a convolutional attention mechanism, is proposed to im- prove the network feature extraction efficiency and reduce the network complexity. First, the convolutional block attention module (CBAM) was introduced, which can strengthen the extraction of useful features from the data, thereby improving the network feature extraction efficiency. Second, the residual structure was optimized, and the original multi-layer convolution was simplified to a single-layer convolution, which reduces the complexity of the model and improves the diagnostic efficiency. Then, the Rectified Linear Unit (ReLU) activation function in the model was replaced with the better-performing Exponential Linear Unit (ELU) activation function. Finally, the overall structure of the ResNet model was optimized. The CBAM-ResNet network model was based on the traditional ResNet18 model and consists of a CBAM module and the optimized ResNet18 model. Finally, experiments were performed using sensor data collected from rolling bearing test benches. The experimental results show that the feature extraction capability and efficiency of the new model are better than those of several other classic deep network models and the original network model, and this reduces the time complexity and model size while also maintaining a high accuracy rate.

Batch Normalization (BN) (Ioffe and Szegedy 2015) normalizes the features of an input image via statistics of a batch of images and hence BN will bring the noise to the gradient of training loss. Previous works indicate that the noise is important for the optimization and generalization of deep neural networks, but too much noise will harm the performance of networks. In our paper, we offer a new point of view that the self-attention mechanism can help to regulate the noise by enhancing instance-specific information to obtain a better regularization effect. Therefore, we propose an attention-based BN called Instance Enhancement Batch Normalization (IEBN) that recalibrates the information of each channel by a simple linear transformation. IEBN has a good capacity of regulating the batch noise and stabilizing network training to improve generalization even in the presence of two kinds of noise attacks during training. Finally, IEBN outperforms BN with only a light parameter increment in image classification tasks under different network structures and benchmark datasets.

Convolutional neural networks (CNN) have achieved outstanding performance in image recognition tasks, but many hand-crafted features still play important roles in some areas. Hand-crafted features are designed to describe image content from specific aspects, which may provide complementary information for CNN in image classification tasks. This paper explores feature fusion methods and proposes a novel framework for combining CNN with hand-crafted features. The framework has two main advantages. First, feature encoder can encode non-normalized features in CNN, which takes advantage of some good edge, texture and local features. Second, joint training strategy makes features fuse better in CNN. We validate that many handcrafted features help to improve the performance of origin CNN. Experiments show our method outperforms the origin CaffeNet on Cifar10 dataset with 79.16% accuracy.

The increasing global population has escalated the demand for fish products. This calls for a stable supply which can only be met through improved aquaculture practices. The automatic recognition of fish diseases from diseased underwater images is one of such practices that aims to control disease, improve fish production and optimize profits. However, due to lack of public fish disease dataset, there has been limited research towards fish disease recognition. Moreover, due to low quality of underwater images and complex underwater environments, traditional hand-designed feature extraction methods or convolutional neural networks (CNNs)-based classifiers cannot adequately recognize fish diseases in real underwater scenes. Therefore, this paper proposes a novel hybrid approach based on multilayer fusion, attention mechanism and online sequential extreme learning machine (OSELM) to recognize fish diseases in aquaculture. We further compare the classification performance of the proposed model with baseline, attention-based, ConvNeXt and Swin transformer models. Feature extraction is enhanced by integrating same level features and focusing on salient features for fish disease recognition. The characteristic information of fish disease is refined by using strongly discriminative features of the infected fish regions and weakening regions of low interest using convolutional block attention module (CBAM). The module is added to the multilayer fusion network to sequentially infer attention maps along the channel and spatial dimensions for every intermediate feature map. Fish disease recognition is then done using OSELM for faster learning and improved classification performance. The models are trained, validated and tested on a custom dataset with image samples collected from various internet sources. The proposed method achieves 94.28% of accuracy, precision of 92.67%, recall of 92.17% and 92.42% of F1- score on dataset with background elimination. The proposed method can be used for fish disease identification in complex underwater environments in aquaculture.

The integration of Multi-layer Perceptrons (MLPs), Convolutional Neural Networks (CNNs), and attention mechanisms has been demonstrated to significantly enhance model performance across various computer vision tasks. In this paper, a novel lightweight neural network architecture, FasterMLP, is proposed to achieve high computational efficiency and accuracy, particularly in resource-constrained and real-time applications. FasterMLP is designed to combine the local connectivity and weight-sharing properties of CNNs with the global feature representation capabilities of MLPs, while feature extraction is enhanced through the Convolutional Block Attention Module and spatial dimensions are effectively reduced using Haar wavelet downsampling without sacrificing critical feature information. The architecture, structured into four stages, has been rigorously evaluated on multiple benchmarks. On the ImageNet-1K dataset, a top-1 accuracy 3.9% higher than that of MobileViT-XXS is achieved by FasterMLP-S, while being 2 and 2.7 faster on GPU and CPU, respectively. On the COCO dataset, the performance of FasterMLP-L is shown to be comparable to FasterNet-L with significantly fewer parameters, and on the Cityscapes dataset, a mean Intersection-over-Union of 81.7% is achieved, surpassing existing methods such as CCNet and DANet. These results demonstrate that FasterMLP can effectively balance computational efficiency and accuracy, making it particularly suitable for visual perception tasks in resource-constrained and real-time environments such as autonomous driving. Code is available at https://github.com/windisl/FasterMLP.

Convolutional neural networks (CNN) have been successfully used in the classification of medical images. However, CNNs extract the features of the image, neglecting the spatial information between the objects since they evaluate each object separately and this makes it difficult to analyse images reliably. CapsNets partially overcomes these limitations by encapsulating the objects in the image and their feature parameters without losing the spatial information between objects. However, this advantage of CapsNet can become a disadvantage due to the fact that CapsNet stores a lot of information, which can lead to an increase in the number of parameters and the complexity of the network. In this study, a new method is proposed in which residual block and CBAM (Convolutional Block Attention Module) modules are used hybridly with CapsNet in order to classify medical images with high accuracy and reliability and, at the same time, to overcome this disadvantage of CapsNet by reducing the number of parameters. In this study, more features of the image are extracted by using the residual block instead of the convolutional layer of CapsNet. The attention mechanism is used to increase the effectiveness of important features and reduce the effect of unnecessary features. In addition, the residual block and CBAM module reduced the number of parameters in CapsNet by approximately 2 million. Brain tumour images and OCT (Optical Coherence Tomography) images were classified with the proposed model. The results of the study show that the proposed model achieves significant results when compared to other CapsNet-based studies in the literature.

Retinal diseases are the leading causes of visual impairment, and early diagnosis is essential for treatment. Optical coherence tomography (OCT), a non-invasive imaging technique, provides high-resolution images for retinal disease classification; however, its complexity and the limitations of manual diagnosis require efficient automated classification methods. To enableearly clinical diagnosis, this paper proposes a Convolutional Block Attention Module (CBAM)- Automatic Mixing Precision (AMP) network (CANet) for automated classification of retinal images. The model categorizes diabetic macular oedema (DME), choroidal neovascularisation (CNV), drusen. and normal cases, which is built upon the ResNet-34 architecture. CBAM is introduced into the residual block to propose the CBAM-Block residual block embedded in the ResNet-34 network model, which combines the channel and spatial attention mechanism to enhance the lesion feature extraction capability. AMP is used to accelerate the training and combine with transfer learning to enhance the model generalization. Meanwhile, median filtering, normalization, dynamic thresholding to remove white edges and data enhancement are used to optimize data quality and alleviate the problem of category imbalance. Classification experiments were performed on the OCT-2017 dataset for the four categories and ablation experiments were performed to demonstrate their effectiveness. The total classification accuracy of the model reaches 0.9890, the AUC value of all categories is 1, where the recall of CNV reaches 1. CBAM, AMP, and transfer learning improve the classification accuracy by 0.9%, 1.6%, and 9.4%, respectively, and the ablation experiments likewise prove that the model remains highly robust to noisy data. The experimental results show that the CANet model significantly improves the OCT image classification performance through multi-module integration, which provides an efficient and reliable technical solution for the automated diagnosis of retinal diseases.

For breast cancer, a malignant tumor developing from breast epithelial tissue, the limitations of traditional diagnostic methods (e.g., diagnostic errors and invasiveness) create critical challenges, underscoring the urgent need for artificial intelligence-assisted technological development. This paper systematically reviews the applications of Convolutional Neural Networks (CNNs) across the entire workflow of breast cancer, including screening, diagnosis, and prognosis, with a focus on hybrid CNNs such as architectures combining Transformer/Long Short-Term Memory (LSTM) and CNN-Support Vector Machine (SVM) models and attention-based CNNs such as contour-enhanced attention and cross-attention mechanisms. It analyzes how these models automate feature extraction from medical images to achieve breast lesion detection, benign-malignant differentiation, and prognosis prediction. Results show that hybrid models like Fusion of Hybrid Deep Features (FHDF) achieve over 98% classification accuracy on datasets such as MIAS by fusing features from multiple CNNs, while attention-based models like Convolutional Block Attention Module (CBAM)-Xception attain an Area Under Curve (AUC) of 0.97 and an accuracy of 89.1% in differentiating benign and malignant lesions. However, challenges remain, including insufficient interpretability, cross-institutional data heterogeneity, and privacy risks. The study proposes integrating medical expert systems and applying transfer learning and domain adaptation techniques to enhance model reliability and generalizability, promoting the translation of CNN technologies into clinical practice and constructing a precise and trustworthy AI-driven breast cancer diagnosis and treatment system.

PurposeThe purpose of this study is to explore the potential of trainable activation functions to enhance the performance of deep neural networks, specifically ResNet architectures, in the task of image classification. By introducing activation functions that adapt during training, the authors aim to determine whether such flexibility can lead to improved learning outcomes and generalization capabilities compared to static activation functions like ReLU. This research seeks to provide insights into how dynamic nonlinearities might influence deep learning models' efficiency and accuracy in handling complex image data sets.Design/methodology/approachThis research integrates three novel trainable activation functions – CosLU, DELU and ReLUN – into various ResNet-n architectures, where “n” denotes the number of convolutional layers. Using CIFAR-10 and CIFAR-100 data sets, the authors conducted a comparative study to assess the impact of these functions on image classification accuracy. The approach included modifying the traditional ResNet models by replacing their static activation functions with the trainable variants, allowing for dynamic adaptation during training. The performance was evaluated based on accuracy metrics and loss profiles across different network depths.FindingsThe findings indicate that trainable activation functions, particularly CosLU, can significantly enhance the performance of deep learning models, outperforming the traditional ReLU in deeper network configurations on the CIFAR-10 data set. CosLU showed the highest improvement in accuracy, whereas DELU and ReLUN offered varying levels of performance enhancements. These functions also demonstrated potential in reducing overfitting and improving model generalization across more complex data sets like CIFAR-100, suggesting that the adaptability of activation functions plays a crucial role in the training dynamics of deep neural networks.Originality/valueThis study contributes to the field of deep learning by introducing and evaluating the impact of three novel trainable activation functions within widely used ResNet architectures. Unlike previous works that primarily focused on static activation functions, this research demonstrates that incorporating trainable nonlinearities can lead to significant improvements in model performance and adaptability. The introduction of CosLU, DELU and ReLUN provides a new pathway for enhancing the flexibility and efficiency of neural networks, potentially setting a new standard for future deep learning applications in image classification and beyond.