Spatial Attention for Pedestrian Detection

Sheep behavior recognition helps to monitor the health status of sheep and prevent the outbreak of infectious diseases. Aiming at the problems of low detection accuracy and slow speed due to the crowding of sheep in real farming scenarios, which can easily obscure each other, this study proposes a lightweight sheep behavior recognition model based on the YOLOv8n model. First, the Convolutional Block Attention Module (CBAM) is introduced and improved in the YOLOv8n model, and the channel attention module and spatial attention module are changed from serial to parallel to construct a novel attention mechanism, PCBAM, to enhance the network’s attention to the sheep and eliminate redundant background information; second, the ordinary convolution in the backbone network is replaced with depth-separable convolution, which effectively reduces the number of parameters in the model and reduces the computational complexity. The study takes the housed breeding sheep as the test object, installs a camera diagonally above the sheep pen to collect images and makes a data set for testing, and in order to verify the superiority of the PD-YOLO model, compares it with a variety of target detection models. The experimental results show that the mean average precision (mAP) of the model proposed in this paper are 95.8%, 98.9%, and 96.2% for the three postures of sheep lying, feeding, and standing, respectively, which are 8.5%, 0.8%, and 0.8% higher than those of YOLOv8n, respectively, and the size of the model has been reduced by 13.3% and the amount of computation has been reduced by 12.1%. The inference speed reaches 52.1 FPS per second, which is better than other models in meeting the real-time detection requirement. To verify the practicality of this research method, the PD-YOLO model was deployed on the RK3399Pro development board for testing, and a high inference speed was achieved. It can provide effective technical support for sheep smart farming.

FBSA-Net: A novel model based on attention mechanisms for emotion recognition in VR and 2D scenes

Case instance segmentation of small farmland based on Mask R-CNN of feature pyramid network with double attention mechanism in high resolution satellite images

In recent years, the performance of the convolutional neural network-based pedestrian detection method has improved significantly. However, an imbalance remains between detection accuracy and speed. In this paper, we employ a one-stage object detection framework and propose a pedestrian detection method based on the multi-scale attention mechanism of a convolutional neural network to improve the imbalance between accuracy and speed. First, a multi-scale convolution module is designed to extract corresponding features at different scales. Second, using the attention module, association information between features is mined from space and channel perspectives to strengthen the original features. Then, the enhanced features are passed through a classification and regression module to perform object positioning and bounding box regression. Finally, to learn more pedestrian location information, we improve the loss function to realise better network training. The proposed method achieved considerable results on the challenging CityPersons and Caltech pedestrian detection datasets.

In recent years, the research of deep learning has received extensive attention, and many breakthroughs have been made in various fields. On this basis, a neural network with the attention mechanism has become a research hotspot. In this paper, we try to solve the image classification task by implementing channel and spatial attention mechanism which improve the expression ability of neural network model. Different from previous studies, we propose an attention module consisting of channel attention module (CAM) and spatial attention module (SAM). The proposed module derives attention graphs from channel dimension and spatial dimension respectively, then the input features are selectively learned according to the importance of the features. Besides, this module is lightweight and can be easily integrated into image classification algorithms. In the experiment, we combine the deep residual network model with the attention module and the experimental results show that the proposed method brings higher image classification accuracy. The channel attention module adds weight to the signals on different convolution channels to represent the correlation. For different channels, the higher the weight, the higher the correlation which required more attention. The main function of spatial attention is to capture the most informative part in the local feature graph, which is a supplement to channel attention. We evaluate our proposed module based on the ImageNet-1K and Cifar-100 respectively. Through a large number of comparative experiments, our proposed model achieved outstanding performance.

In this paper, we propose a novel fusion framework based on a dense network with channel and spatial attention (CSpA-DN) for PET and MR images. In our approach, an encoder composed of the densely connected neural network is constructed to extract features from source images, and a decoder network is leveraged to yield the fused image from these features. Simultaneously, a self-attention mechanism is introduced in the encoder and decoder to further integrate local features along with their global dependencies adaptively. The extracted feature of each spatial position is synthesized by a weighted summation of those features at the same row and column with this position via a spatial attention module. Meanwhile, the interdependent relationship of all feature maps is integrated by a channel attention module. The summation of the outputs of these two attention modules is fed into the decoder and the fused image is generated. Experimental results illustrate the superiorities of our proposed CSpA-DN model compared with state-of-the-art methods in PET and MR images fusion according to both visual perception and objective assessment.

In recent years, attention mechanisms have shown great potential in various computer vision tasks. However, most existing methods focus on developing more complex attention modules for better performance, which inevitably increases the complexity of the model. To overcome performance and complexity tradeoffs, this paper proposes efficient spatial and channel attention (ESCA), a symmetric, comprehensive, and efficient attention module. By analyzing squeeze-and-excitation (SE), convolutional block attention module (CBAM), coordinate attention (CA), and efficient channel attention (ECA) modules, we abandon the dimension-reduction operation of SE module, verify the negative impact of global max pooling (GMP) on the model, and apply a local cross-channel interaction strategy without dimension reduction to learn attention. We not only care about the channel features of the image, we also care about the spatial location of the target on the image, and we take into account the effectiveness of channel attention, so we designed the symmetric ESCA module. The ESCA module is effective, as demonstrated by its application in the ResNet-50 classification benchmark. With 26.26 M parameters and 8.545 G FLOPs, it introduces a mere 0.14% increment in FLOPs while achieving over 6.33% improvement in Top-1 accuracy and exceeding 3.25% gain in Top-5 accuracy. We perform image classification and object detection tasks on ResNet, MobileNet, YOLO, and other architectures on popular datasets such as Mini ImageNet, CIFAR-10, and VOC 2007. Experiments show that ESCA can achieve great improvement in model accuracy at a very small cost, and it performs well among similar models.

The segmentation and classification of different types of nuclei plays an important role in discriminating and diagnosing of the initiation, development, invasion, metastasis and therapeutic response of tumors of various organs. Recently, deep learning method based on attention mechanism has achieved good results in nuclei semantic segmentation. However, the design of attention module architecture relies heavily on the experience of researchers and a large number of experiments. Therefore, in order to avoid this manual design and achieve better performance, we propose a new Neural Architecture Search-based Spatial and Channel joint Attention Module (NAS-SCAM) to obtain better spatial and channel weighting effect. To the best of our knowledge, this is the first time to apply NAS to the attention mechanism. At the same time, we also use synchronous search strategy to search architectures independently for different attention modules in the same network structure. We verify the superiority of our methods over the state-of-the-art attention modules and networks in public dataset of MoNuSAC 2020. We make our code and model available at https://github.com/ZuhaoLiu/NAS-SCAM.

In this paper, we focus on the semantic image synthesis task that aims at transferring semantic label maps to photo-realistic images. Existing methods lack effective semantic constraints to preserve the semantic information and ignore the structural correlations in both spatial and channel dimensions, leading to unsatisfactory blurry and artifact-prone results. To address these limitations, we propose a novel Dual Attention GAN (DAGAN) to synthesize photo-realistic and semantically-consistent images with fine details from the input layouts without imposing extra training overhead or modifying the network architectures of existing methods. We also propose two novel modules, i.e., position-wise Spatial Attention Module (SAM) and scale-wise Channel Attention Module (CAM), to capture semantic structure attention in spatial and channel dimensions, respectively. Specifically, SAM selectively correlates the pixels at each position by a spatial attention map, leading to pixels with the same semantic label being related to each other regardless of their spatial distances. Meanwhile, CAM selectively emphasizes the scale-wise features at each channel by a channel attention map, which integrates associated features among all channel maps regardless of their scales. We finally sum the outputs of SAM and CAM to further improve feature representation. Extensive experiments on four challenging datasets show that DAGAN achieves remarkably better results than state-of-the-art methods, while using fewer model parameters.

This paper examines the use of deep learning methods for accurately segmenting breast cancer in ultrasound images, which is an important step in computer-aided cancer diagnosis. To improve border location and region-level accuracy, the proposed framework presents a hybrid attention-based segmentation model, combining the DenseNet121 encoder and three specially designed attention modules: global spatial attention (GSA), temporary spatial attention (TSA) and spatial feature enhancement block (SFEB). Its unique approach is a multi-level attention integration strategy that improves both local features and global awareness, allowing models to understand complex breast injuries more effectively. After training and evaluating the model using the Breast Ultrasound pictures (BUSI) dataset, which comprises 780 grayscale ultrasound pictures with matching binary masks and labels such as benign, malignant, or normal. With a mean Intersection over Union (IoU) of 0.94, overall segmentation accuracy of 0.89, Dice coefficient of 0.94, precision of 0.93, and recall of 0.94, the suggested method outperforms traditional Convolutional Neural Network (CNN) based techniques and shows great promise for supporting radiological diagnoses.

The three projects documented in this thesis investigated the role of three forms of attention in visual processing. Spatial, feature-based and temporal attention. The spatial and feature-based attentional characteristics were studied by conducting extracellular recordings from neurons of area MT and MSTd of macaque visual cortex. While the role of temporal attention was studied in a human psychophysics project. In the physiology projects we first defined the tuning characteristics of neurons from area MT and MSTd for two stimulus dimensions, namely spiral motion stimuli (SMS) and linear motion stimuli (LMS) on the basis of directionality index, the relative frequency of tuned versus untuned neurons, and the individual cells responses to the two forms of stimuli. We found that MT and MSTd showed differential tuning to LMS and SMS. Area MSTd demonstrated better selectivity for SMS as compared to LMS. Conversely, area MT was better tuned to LMS than SMS. Further, we compared spatial and feature-based attentional modulation in area MT and MSTd across SMS and LMS. We found that spatial attention modulates the responses of neurons in both the areas irrespective of the feature dimension defining the attended stimuli. On the other hand feature-based attention was restricted to the stimulus dimension for which an area showed better selectivity, that is SMS for MSTd and LMS for MT. We also observed that the correlation of firing rates with reaction times was restricted to an area’s preferred stimulus dimension. The presence of a significant negative correlation of firing rate with reaction time in area MSTd when attention was directed to its preferred dimension (SMS), indicated that these neurons were involved in perceptual processing of SMS. Similarly, the presence of a significant negative correlation of firing rates with the reaction times in area MT when attention was directed to LMS, provides evidence that MT neurons were involved in perceptual processing of LMS. Taken together, these results show that neurons across the visual cortex are tuned to a range of stimulus dimensions. This multiple stimulus selectivity is thought to be utilized to develop a saliency map (by interaction of bottom-up and top-down processes), which marks the behaviorally relevant aspects of visual space. These behaviorally relevant aspects of the visual space are then preferentially processed throughout the visual hierarchy, and our results suggests that a given area contributes to visual perception by flexibly processing only the stimulus dimension for which it shows best selectivity as a function of stimulus relevance. Finally in human subjects we investigated the temporal characteristics of visual motion processing, using the ‘attentional blink’ paradigm. Attentional blink is a widely studied phenomenon, but almost all studies to the date have been are restricted to stationary stimuli. We here extended this paradigm to motion stimuli. We found the presence of an attentional blink in the motion domain, in that the detection of second of the two target (motion) stimuli was severely degraded when it was presented within 450ms of the first target (motion) stimulus. However, our results were slightly different from the experiments with the stationary stimuli. First, there was an absence of the lag-1 sparing effect typically found with stationary stimuli and second, the time interval over which the attentional blink was observed was about 100ms shorter than typically reported for stationary stimuli. Post-hoc analysis showed that these effects in the motion domain were dependent upon the direction of motion of the first target. Our data showed that when the first target moved in the opposite direction to the second target, there was no attentional blink, but when the first target moved in an orthogonal direction to the second target, there was an attentional blink. These results are in agreement to previous studies showing that the processing of motion stimuli is facilitated when it is preceded by a stimulus moving in opposite directions, while processing of motion stimuli is suppressed when it is preceded by a stimulus moving in an orthogonal direction. Taken together the results of this thesis provides an important contribution to our understanding of the role of attention in visual motion processing. It will be interesting to extend this study in other visual cortical areas and measure spatial and feature-based attention from same set of neurons across different stimulus dimension.

Brain tumors are the result of irregular development of cells. It is a major cause of adult demise worldwide. Several deaths can be avoided with early brain tumor detection. Magnetic resonance imaging (MRI) for earlier brain tumor diagnosis may improve the chance of survival for patients. The most common method of diagnosing brain tumors is MRI. The improved visibility of malignancies in MRI makes therapy easier. The diagnosis and treatment of brain cancers depend on their identification and treatment. Numerous deep learning models are proposed over the last decade including Alexnet, VGG, Inception, ResNet, DenseNet, etc. All these models are trained on a huge dataset, ImageNet. These general models have many parameters, which become irrelevant when implementing these models for a specific problem. This study uses a custom deep-learning model for the classification of brain MRIs. The proposed Disease and Spatial Attention Model (DaSAM) has two modules; (a) the Disease Attention Module (DAM), to distinguish between disease and non-disease regions of an image, and (b) the Spatial Attention Module (SAM), to extract important features. The experiments of the proposed model are conducted on two multi-class datasets that are publicly available, the Figshare and Kaggle datasets, where it achieves precision values of 99% and 96%, respectively. The proposed model is also tested using cross-dataset validation, where it achieved 85% accuracy when trained on the Figshare dataset and validated on the Kaggle dataset. The incorporation of DAM and SAM modules enabled the functionality of feature mapping, which proved to be useful for the highlighting of important features during the decision-making process of the model.

Attentional modulation of motion-induced blindness

In recent years, vein biometrics has gained significant attention due to its high security and privacy features. While deep neural networks have become the predominant classification approaches for their ability to automatically extract discriminative vein features, they still face certain drawbacks: 1) Existing transformer-based vein classifiers struggle to capture interactive information among different attention modules, limiting their feature representation capacity; 2) Current label enhancement methods, although effective in learning label distributions for classifier training, fail to model long-range relations between classes. To address these issues, we present ALE-IVT, an Attention Label Enhancement-based Interactive Vein Transformer for palm-vein recognition. First, to extract vein features, we propose an interactive vein transformer (IVT) consisting of three branches, namely spatial attention, channel attention, and convolutional module. In order to enhance performance, we integrate an interactive module that facilitates the sharing of discriminative features among the three branches. Second, we explore an attention-based label enhancement (ALE) approach to learn label distribution. ALE employs a self-attention mechanism to capture correlation between classes, enabling the inference of label distribution for classifier training. As self-attention can model long-range dependencies between classes, the resulting label distribution provides enhanced supervised information for training the vein classifier. Finally, we combine ALE with IVT to create ALE-IVT, trained in an end-to-end manner to boost the recognition accuracy of the IVT classifier. Our experiments on three public datasets demonstrate that our IVT model surpasses existing state-of-the-art vein classifiers. In addition, ALE outperforms current label enhancement approaches in term of recognition accuracy.

Impurity monitoring study for corn kernel harvesting based on machine vision and CPU-Net