Discriminative correlation tracking based on spatial attention mechanism for low-resolution imaging systems

Liver cancer, especially hepatocellular carcinoma (HCC), remains one of the most fatal cancers globally, emphasizing the critical need for accurate tumor segmentation to enable timely diagnosis and effective treatment planning. Traditional imaging techniques, such as CT and MRI, rely on manual interpretation, which can be both time-intensive and subject to variability. This study introduces FasNet, an innovative hybrid deep learning model that combines ResNet-50 and VGG-16 architectures, incorporating Channel and Spatial Attention mechanisms alongside Monte Carlo Dropout to improve segmentation precision and reliability. FasNet leverages ResNet-50’s robust feature extraction and VGG-16’s detailed spatial feature capture to deliver superior liver tumor segmentation accuracy. Channel and spatial attention mechanisms could selectively focus on the most relevant features and spatial regions for suitable segmentation with good accuracy and reliability. Monte Carlo Dropout estimates uncertainty and adds robustness, which is critical for high-stakes medical applications. Tested on the LiTS17 dataset, FasNet achieved a Dice Coefficient of 0.8766 and a Jaccard Index of 0.8487, surpassing several state-of-the-art methods. The Channel and Spatial Attention mechanisms in FasNet enhance feature selection, focusing on the most relevant spatial and channel information, while Monte Carlo Dropout improves model robustness and uncertainty estimation. These results position FasNet as a powerful diagnostic tool, offering precise and automated liver tumor segmentation that aids in early detection and precise treatment, ultimately enhancing patient outcomes.

Image dehazing is a useful technique which can eliminate the bad effect of haze on images and enhance the performances of image/video processing algorithms in hazy weather. In this chapter, we first propose a single-image dehazing method. We estimate the initial transmission properly based on latent region segmentation and refine the estimated initial transmission by an objective function with a novel weighted L1-norm regularization term. The half-quadratic splitting minimization method is employed to solve this optimization problem. We also define an evaluation function to estimate the reliable global atmospheric light. With the refined transmission map and atmospheric light, we recover the haze-free image by the haze imaging model. We then propose a retinex-based decomposition model for a hazy image and a novel end-to-end image dehazing network. In the model, the illumination of the hazy image is decomposed into natural illumination for the haze-free image and residual illumination caused by haze. Based on this model, we design a deep retinex dehazing network (RDN) to jointly estimate the residual illumination map and the haze-free image. Our RDN consists of a multiscale residual dense network and a U-Net with channel and spatial attention mechanisms. The multiscale residual dense network can simultaneously capture global contextual information from small-scale receptive fields and local detailed information from large-scale receptive fields to precisely estimate the residual illumination map caused by haze. And in the dehazing U-Net, we apply the channel and spatial attention mechanisms in the skip connection to achieve a trade-off between overdehazing and underdehazing by automatically adjusting the channel-wise and pixel-wise attention weights so as to obtain a refined haze-free image.

Real-world License plate detections presents notable challenges arising due to some factors such as variable illumination, occlusions, complex backgrounds and plate distortions. Traditional approaches such as edge detection often generate numerous false positives under such conditions, while template matching methods typically struggle with adaptability to diverse plate styles. Although Convolutional Neural Networks (CNNs) have improved feature extraction capabilities, it may still misallocate attention to irrelevant regions, reducing detection accuracy. To overcome these limitations, this study proposes an refined deep learning architecture that integrates Convolutional Block Attention Modules (CBAM) into CNN-based feature extractors. By jointly exploiting both channel and spatial attention mechanisms, the model selectively highlights salient features and minimizes the influence of background noise, enhancing its focus on license plate regions across varied image environment. Experimental evaluation using Kaggle License Plate Dataset shows that the proposed CNN + CBAM framework achieves a precision of 92.41%, recall of 89.73%, F1-score of 91.05%, and mAP@0.5 of 88.9%. These results represents an improvement of 7.82% in F1-score and 10.25% in mAP compared to the baseline CNN, validating that the extracted crucial features substantially improve detection accuracy while minimizing false detections. The proposed approach also demonstrates a significantly reduced false detection rate of 5.6%, outperforming both traditional techniques and conventional CNNs based models.

Object tracking is one of the important research topics in the direction of computer vision. And occlusion, deformation and scale change are the most challenging issues in the realm of tracking. In an effort to address the issue that the Siamese network-based object tracking algorithm’s resilience and success rate are both low and weak in complicated circumstances such as similar object interference and object occlusion, a Siamese network Object tracking algorithm combined with attention mechanism based on the existing Siamese network algorithm is proposed. Firstly, in order to extract the features of the template picture and the sample picture, the modified AlexNet convolutional neural network <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[1]</sup> is utilized as the feature extraction network. Secondly, in order to increase the model’s precision and increase tracking’s rate of success, the module for the attention mechanism, which consists of the channel attention mechanism and the spatial attention mechanism, is introduced. The channel attention mechanism can focus on the information useful for tracking the Object, suppress unimportant background information, and the spatial attention mechanism is capable of picking up on the object’s abundant spatial information. Experiments show that the proposed Siamese network target tracking algorithm combined with attention mechanism achieves contestable results on the OTB100 test dataset, and the enhanced method outperforms the original approach in terms of performance.

Ship monitoring using Synthetic Aperture Radar (SAR) data faces significant challenges in detecting small vessels due to low spatial resolution and speckle noise. While ESRGAN (Enhanced Super-Resolution Generative Adversarial Network) has shown promise for image super-resolution, it struggles with SAR imagery characteristics. This study proposes SA/SU-ESRGAN, which extends the SU-ESRGAN framework by incorporating a spatial attention mechanism loss function. SU-ESRGAN introduced semantic structural loss to accurately preserve ship shapes and contours; our enhancement adds spatial attention to focus reconstruction efforts on ship regions while suppressing background noise. Experimental results demonstrate that SA/SU-ESRGAN successfully detects small vessels that remain undetectable by SU-ESRGAN, achieving improved detection capabilities with a PSNR of approximately 26 dB (SSIM is around 0.5) and enhanced visual clarity in ship boundaries. The spatial attention mechanism effectively reduces noise influence, producing clearer super-resolution results suitable for maritime surveillance applications. Based on the HRSID dataset, a representative dataset for evaluating ship detection performance using SAR data, we evaluated ship detection performance using images in which the spatial resolution of the SAR data was artificially degraded using a smoothing filter. We found that with a 4 × 4 filter, all eight ships were detected without any problems, but with an 8 × 8 filter, only three of the eight ships were detected. When super-resolution was applied to this, six ships were detected.

Accurately predicting trajectory of vehicles is a key capability for autonomous vehicles during real-world driving scenarios, which requires the autonomous vehicles to perceive the surrounding environment and analyze the track histories to judge the driving intentions of the focused surrounding vehicles in order to make decisions for the next trajectory route. In this paper, we propose a hybrid attentional trajectory prediction model incorporating both temporal attentional and spatial attentional mechanisms, which has a temporal attentional model that can be used to capture important temporal information affecting trajectory prediction and a spatial attentional mechanism that can better introduce spatial dependencies and potential driving intention information of surrounding and target vehicles. The models are evaluated for performance on publicly available highD and NGSIM datasets. By comparing the results with other state-of-the-art models for trajectory prediction, it is found that the model achieves an improvement in prediction performance, indicating that this model can more accurately learn driving information from the spatio-temporal perspective of the target driving vehicle’s attention. Further, by qualitatively analyzing the distribution of attention weights from the spatio-temporal perspective, we assessed the interpretability of the model on the maneuverability judgment of the vehicle.

Physical rehabilitation is a medical specialty that focuses on the restoration of body functions in the safest and most effective way possible. During a rehabilitation exercise session, patients' behavior reflects their health status and is an important indicator of the treatment outcome. Building an automatic system for the evaluation of human motion quality in an objective and reliable way can be used in medicine to establish a differential diagnosis, to choose the adequate treatment, or for patient monitoring. Deep learning has become the state-of-the-art in Human Action and Human Behavior Recognition from videos. Most of the state-of-the-art model architectures are CNNs based and often use RNNs to calculate temporal dependencies among frames, and ignore the topological structure of the human body. In this work, we propose to use a Graph Transformer network with spatial and temporal attention mechanisms for physical rehabilitation evaluation. First, we used a standard Transformer network with a selfattention mechanism, then we take advantage of graph skeletons as inputs for a two-stream spatio-temporal graph network with both spatial and temporal attention mechanisms. We used our model on UI-PRMD, a benchmark dataset that provides skeleton data using motion capture systems. Our results show that our attention-based based ST-GCN models outperform the state-of-the-art methods on quality score prediction and binary classification.

Impervious surface extraction from remote sensing images has become a promising technology to measure the urban ecological environment and monitor human activity. However, due to the complex characteristics of impervious landscapes, most researches on impervious surface extraction hardly identify the scattered and small objects especially in difficult areas, which severely affect the accuracy of mapping impervious surface. In this work, we propose a channel and spatial attention mechanism (CSAM) to extract impervious surface in difficult areas, which includes a channel attention module to learn the relationship in the multi-channel remote sensing images and a spatial attention module to capture the features of the inconspicuous objects. Experiments with the Sentinel-2 dataset in South Africa demonstrate that CSAM can outperform the state-of-the-art methods.

The reconstruction of the self-luminous image of a flame through deep learning can inform research on the characteristics of combustion of a scramjet. In this study, the authors propose a residual network model based on the channel and spatial attention mechanisms to reconstruct the self-luminous image of a flame from schlieren images of the flow field of a scramjet. We compare the reconstruction-related performance of single-path and dual-path models under different conditions. The channel and spatial attention mechanisms enable the model to focus on important feature-related information, and the residual connection prevents gradient disappearance to improve the capability of the model for generalization. The proposed method was tested through a supersonic combustion experiment in a ground wind tunnel under different equivalence ratios, and data on the flow field of the combustion chamber and the evolution of the flame were recorded as a dataset. A number of experiments as well as subjective and objective analyses were subsequently carried out on this dataset. The results show that the effect of reconstruction is consistent with the original image of the flame, and the geometric characteristics of the flame are accurately reconstructed.

Pneumonia remains a global health issue, creating the need for accurate detection methods for effective treatment. Deep learning models like ResNet50 show promise in detecting pneumonia from chest X-rays; however, their black-box nature limits the transparency, which fails to meet that needed for clinical trust. This study aims to improve model interpretability by comparing four interpretability techniques, which are Layer-wise Relevance Propagation (LRP), Adversarial Training, Class Activation Maps (CAMs), and the Spatial Attention Mechanism, and determining which fits best the model, enhancing its transparency with minimal impact on its performance. Each technique was evaluated for its impact on the accuracy, sensitivity, specificity, AUC-ROC, Mean Relevance Score (MRS), and a calculated trade-off score that balances interpretability and performance. The results indicate that LRP was the most effective in enhancing interpretability, achieving high scores across all metrics without sacrificing diagnostic accuracy. The model achieved 0.91 accuracy and 0.85 interpretability (MRS), demonstrating its potential for clinical integration. In contrast, Adversarial Training, CAMs, and the Spatial Attention Mechanism showed trade-offs between interpretability and performance, each highlighting unique image features but with some impact on specificity and accuracy.

In today's logistics and delivery landscape, mobile robot delivery systems have attracted considerable attention due to their efficiency and adaptability. Nevertheless, current robotic delivery solutions encounter various obstacles in complex and dynamically changing environments. Traditional algorithms, for instance, struggle with processing high-dimensional and unstructured data, resulting in inefficient adaptation to real-time environmental changes, which compromises accuracy and efficiency in path planning and task execution. Moreover, the lack of robust perception and decision-making mechanisms limits the robots' ability to handle intricate scenarios and fluctuating delivery demands. To tackle these challenges, this paper proposes an optimization approach for mobile robot delivery systems that leverages deep learning. The study initially integrates a spatial attention mechanism within the model, enabling the robot to focus on critical environmental regions and dynamically adjust attention points, thus enhancing obstacle recognition and avoidance in complex settings, which improves navigation accuracy and path planning. Furthermore, the Deep Deterministic Policy Gradient (DDPG) algorithm is utilized to optimize policies, supporting efficient learning in high-dimensional continuous spaces and empowering robots to acquire effective delivery strategies in challenging environments. Finally, an end-to-end optimization approach allows the system to convert sensor inputs directly into control commands, reducing intermediate complexity and minimizing error accumulation, thereby streamlining the system’s structure. Experimental results confirm that the proposed method substantially boosts delivery system performance, excelling in key metrics like path planning accuracy, task efficiency, and system robustness. The successful integration of the spatial attention mechanism with the deep policy gradient algorithm demonstrates a valuable new approach for advancing future robot delivery system optimizations.

Facial expression recognition (FER) in uncontrolled, in-the-wild settings is a challenging task due to variations in pose, illumination, occlusion, and background clutter. The state-of-the-art methods on FER employ fixed size pooling in attention networks, resulting in suboptimal robustness in unconstrained environments. In this work, a novel Multi-Scale Attention Network (MSAN) is integrated with the YOLOv8 backbone architecture for expression classification. The YOLOv8 backbone provides efficient feature extraction with a lightweight, detection-oriented architecture, enabling real-time processing and strong spatial feature representation. The Convolutional Block Attention Mechanism (CBAM) is employed in the proposed work. The standard convolutional blocks in YOLOv8 backbone are replaced with MSAN, wherein the fixed-size pooling in the spatial attention mechanism of CBAM is replaced with a Pyramid Pooling Mechanism (PPM). The PPM employs adaptive average pooling at multiple resolutions to capture multi-scale contextual information, which is subsequently fused into a unified feature representation. This enables the spatial attention mechanism in MSAN to capture both global facial context and local region-level cues simultaneously, leading to superior robustness to scale variations, partial occlusions, and diverse expression variations. The work is evaluated on benchmark wild datasets including RAF-DB, FER2013, and AffectNet, with all experiments performed using 5-fold cross-validation. Experimental results demonstrate that the multi-scale attention network with PPM in YOLOv8 backbone achieved improvement in classification accuracy and robustness when compared with the state-of-the-art methods for expression classification in the wild. These findings highlight the effectiveness of integrating MSAN into lightweight detection-based backbones for real-world FER applications.

Infrared and visible image fusion aims to fuse the thermal information of infrared images and the texture information of visible images into images that are more in compliance with people’s visual perception characteristics. However, in the existing related work, the fused images have incomplete contextual information and poor fusion results. This paper presents a new image fusion algorithm—OMOFuse. At first, both the channel and spatial attention mechanisms are optimized by a DCA (dual-channel attention) mechanism and an ESA (enhanced spatial attention) mechanism. Then, an ODAM (optimized dual-attention mechanism) module is constructed to further improve the integration effect. Moreover, a MO module is used to improve the network’s feature extraction capability for contextual information. Finally, there is the loss function ℒ from the three parts of SSL (structural similarity loss), PL (perceptual loss), and GL (gap loss). Extensive experiments on three major datasets are performed to demonstrate that OMOFuse outperforms the existing image fusion methods in terms of quantitative determination, qualitative detection, and superior generalization capabilities. Further evidence of the effectiveness of our algorithm in this study are provided.

Faced with a large number of non-cooperative targets in space, more complete image data can be obtained at a closer distance. Image segmentation can be performed to obtain morphological information on the relevant targets, which can provide reliable data support for subsequent tasks such as situational awareness. In this paper, we improve the Deeplab V3+ network by adding Convolutional Block Attention Module (CBAM), which incorporates the channel attention mechanism and the spatial attention mechanism, and verify that the algorithm has good performance and can be used for practical applications by segmenting the spacecraft attitude estimation dataset from the European Space Agency Space Rendezvous Laboratory.

In order to solve the problems of unbalanced sample data and the lack of consideration of temporal information in existing Siamese-based trackers, this paper proposes a Siamese recurrent neural network and region proposal network (Siamese R-RPN), which can be trained in an end-to-end manner. Siamese R-RPN is consisted of Siamese network, recurrent neural network and region proposal network. Image features extracted by the Siamese network are strengthened by the channel and spatial attention mechanisms, and are sent to the RPN for classification and regression. Temporal information is processed by a recurrent neural network-based Long Short-Term Memory (LSTM) to predict the rough location of the target, it is mapped to the anchor feature map of the RPN for anchor selection. This makes the positive and negative samples participating in the training procedure to become more balanced and representative. Because of the collaborative use of temporal and spatial information, the tracker proposed in this paper has achieved state-of-the-art performance on three large tracking benchmarks&#x2014;OTB 2015, VOT2016 and VOT 2018&#x2014;where this verifies its effectiveness.