Image semantic segmentation based on multi-layer feature information fusion and dual convolutional attention mechanism

Fire and smoke recognition has a wide range of applications in fire warning, anti-jamming, battlefield situational awareness and other fields. Using the method of semantic segmentation for fire and smoke recognition can obtain the fire and smoke area of the image more accurately and comprehensively. Although the DeepLabV3+ network based on the Encoder-decoder structure has achieved excellent performance in semantic segmentation in many fields, it still has some shortcomings, such as slow fitting, blurred edge segmentation, and holes in the feature map. In order to address the deficiencies of DeepLabV3+,In this paper we propose a novel method of introducing parallel dual attention mechanism network(DAMN). We designed a parallel connection between DAMN and atous spatial pyramid pooling(ASPP). DAMN and ASPP process the feature map generated by the network in parallel, and then fuse the feature map information generated by DAMN and ASPP module. Meanwhile, we built a multi-scene fire and smoke dataset named “Fire-Smoke” for comprehensive assessment in fire and smoke image recognition. Fire-Smoke contains over 8,000 hand-labeled smoke labels. We achieve better segmentation performance on Fire-Smoke dataset. In particular, it achieves an average IoU score of 85.77 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">%</sup> on the Fire-Smoke test set, an improvement of 2.18% over the DeepLabV3+.

In computer-aided diagnosis systems, precise feature extraction from CT scans of colorectal cancer using deep learning is essential for effective prognosis. However, existing convolutional neural networks struggle to capture long-range dependencies and contextual information, resulting in incomplete CT feature extraction. To address this, the PEDRA-EFB0 architecture integrates patch embeddings and a dual residual attention mechanism for enhanced feature extraction and survival prediction in colorectal cancer CT scans. A patch embedding method processes CT scans into patches, creating positional features for global representation and guiding spatial attention computation. Additionally, a dual residual attention mechanism during the upsampling stage selectively combines local and global features, enhancing CT data utilization. Furthermore, this paper proposes a feature selection algorithm that combines autoencoders and entropy technology, encoding and compressing high-dimensional data to reduce redundant information and using entropy to assess the importance of features, thereby achieving precise feature selection. Experimental results indicate the PEDRA-EFB0 model outperforms traditional methods on colorectal cancer CT metrics, notably in C-index, BS, MCC, and AUC, enhancing survival prediction accuracy. Our code is freely available at https://github.com/smile0208z/PEDRA .

WiFi-based human activity recognition (HAR) leveraging channel state information (CSI) faces challenges due to multipath effects, leading to ambiguous feature representation. To address this, we propose AMCS-TCN, a model integrating dual attention mechanisms and a multi-channel, multi-scale temporal convolutional network (TCN). This model utilizes spatial diversity from MIMO systems to enhance feature representation. The dual attention module, comprising feature map fusion attention (FMFA) and Feature Channel Attention (FCA), adaptively focuses on discriminative information. Experimental results on a public dataset and real-world environments show an average accuracy of 98.85% and robust performance in complex settings. Our code and models are released at https://github.com/boyuanzhang759-rgb/AMCS-TCN-by-BoYuan for reproduction.

Road segmentation from high-resolution remote sensing images is a hot research topic in recent years. The roads in high-resolution remote sensing images are irregular with large scale variations. In addition, the scenes of such images are complicated with interference factors that have negative impacts on the segmentation results. On this account, we propose a dual attention network (DA-LinkNet), which combines the advantages of D-LinkNet and the dual attention mechanism. The network is built with D-LinkNet architecture and the Coordinate Attention block is added in its center part, which aims to enhance feature representation. To better combine features from different branches, we adopt the Attentional Feature Fusion block to replace the linear feature fusion operation of skip connection. The comparative experiments on DeepGlobe Road Extraction dataset demonstrate the effectiveness and superiority of the proposed method, which exceed state-of-the-art methods by at least 2 % for mean Intersection over Union (mIoU).

Automatic vessel structure segmentation is essential for an automatic disease diagnosis system. The task is challenging due to vessels’ different shapes and sizes across populations. This paper proposes a multiscale network with dual attention to segment various retinal blood vessels. The network injects a spatial attention module and channel attention module on a feature map, whose size is one-eighth of the input size. The network also uses multiscale input to receive multi-level information, and the network uses the multiscale output to gain more supervision. The proposed method is tested on two publicly available datasets: DRIVE and CHASEDB1. The accuracy, AUC, sensitivity, and specificity on the DRIVE dataset are 0.9615, 0.9866, 0.7709, and 0.9847, respectively. On the CHASEDB1 dataset, the metrics are 0.9800, 0.9892, 0.8215, and 0.9877, respectively. The ablative study further shows effectiveness for each part of the network. Multiscale and dual attention mechanism both improve performance. The proposed architecture is simple and effective. The inference time is 12 ms on a GPU and has potential for real-world applications. The code will be made publicly available.

Monitoring the health of rolling mills has always been a concern for the steel industry. However, existing methods make it difficult to obtain fault features under the conditions of unbalanced data and serious disturbances during rolling. Therefore, we propose a fault diagnosis of the rolling mill based on the gramian angular difference field and the dual-attention residual network. Firstly, original one-dimensional vibration signals are converted into two-dimensional GADF images as the input of the network. Secondly, the dual attention mechanism of channel and space is introduced into the residual network to focus on the channel and spatial information of the input image for the re-integration of the extracted features. Finally, the model is trained to complete the recognition of faults. To validate the superiority of the proposed method, datasets with different data distributions were designed on the rolling mill test bench. Experimental comparisons have proven the superiority of GADF and the dual-attention mechanism. Moreover, when compared with other methods on several imbalanced datasets, the proposed method achieves an average accuracy that is at least 1.8% higher than that of other methods.

This paper introduces OVANet, a new deep learning framework for the diagnosis and classification of ovarian cancer subtypes from histopathological images. OVANet has been designed by integrating modified VGG19 and InceptionV3 blocks, incorporating dual attention mechanism (Squeeze-and-Excitation and custom spatial attention) to capture rich, multi-scale, and context-aware feature representations. This unique architecture combines intermediate layers from both base models, along with additional convolutional and max-pooling layers, dual attention mechanisms and global average pooling to extract diverse image features crucial for accurate subtype differentiation. OVANet is trained and evaluated on an original dataset of 508 histopathological images, augmented to approximately 13,024 images through preprocessing and augmentation techniques like rotations, shifts, zooms, and flips. This approach significantly enhances the generalization capabilities of the model, enabling it to achieve superior performance compared to well-known pretrained models and existing research outputs. The propose model shows accuracy, precision and AUC of 99.01% along with a recall of 99.87%. Using feature map visualization of all layers of the model and a model explainibility technique GRADCam, visual representation of classification have been shown which will help to establish transparency in clinical aspect. This architecture can be considered as a significant advancement in medical imaging, especially for the diagnosis and treatment planning of ovarian cancer. To reduce the death rate of cancer, proper detection is very essential and this model exhibits so promising results that can easily implemented clinically.

Data across different domains can be influenced by variations in language styles and expressions, making it challenging to migrate specialized words, particularly when focusing on aspectual words. This complexity poses difficulties in conducting cross-domain aspectbased sentiment analysis. The article begins by introducing BERT for generating word vectors as representations of training texts, enhancing text semantics in the word vector representation stage. To capture more nuanced interaction information and context-related details, the paper proposes the Bert-BiLSTM model with a dual attention mechanism(BBDAM), which divides the original input sequence into three parts: above, aspectual words, and below. A dual attention mechanism was used to assess the interaction of aspect words with the three aspects (above, below, and neighboring words) in the three discourse segments. This mechanism allows for the comprehensive extraction of interaction information. By comparing with other modeling approaches, the experimental results show that the BB-DAM model produces good results in fine-grained cross-domain sentiment analysis.

With the development of airborne light detection and ranging (LiDAR) technology, it has become a common and efficient way to collect large-scale 3D spatial information. However, efficient and automatic semantic segmentation of LiDAR data, in the form of 3D point clouds, remains a persistent challenge. To address this, a dual attention neural network (DA-Net) is proposed, consisting of two different blocks, namely augmented edge representation (AER) and elevation attentive pooling (EAP). First, the AER can adaptively represent local orientation and position, thereby effectively enhancing geometric information. Second, the captured local features of centroid points are utilized to further encode discriminative features using the EAP with the learned attention scores. Finally, a location homogeneity (LH) module is devised to explore the long-range relationship in an encoder-decoder network. Benefiting from the dual attention module, geometric information hidden in unorganized point clouds can be effectively propagated. Besides, the LH forces the network to pay attention to the semantic consistency of elevated objects, which facilitates both point- and object-level point cloud semantic segmentation for scene understanding. A benchmark dataset is used to assess the proposed method, which achieves an overall accuracy of 85.98% and an average F1 score of 72.31%. In addition, comparisons with other latest deep learning methods on the 2019 Data Fusion Contest dataset further demonstrate the robustness and generalization ability of the proposed method.

This paper focuses on the problems of omission, misclassification, and inter-adhesion due to overly dense distribution, intraclass diversity, and interclass variability when extracting winter wheat (WW) from high-resolution images. This paper proposes a deep supervised network RAunet model with multi-scale features that incorporates a dual-attention mechanism with an improved U-Net backbone network. The model mainly consists of a pyramid input layer, a modified U-Net backbone network, and a side output layer. Firstly, the pyramid input layer is used to fuse the feature information of winter wheat at different scales by constructing multiple input paths. Secondly, the Atrous Spatial Pyramid Pooling (ASPP) residual module and the Convolutional Block Attention Module (CBAM) dual-attention mechanism are added to the U-Net model to form the backbone network of the model, which enhances the feature extraction ability of the model for winter wheat information. Finally, the side output layer consists of multiple classifiers to supervise the results of different scale outputs. Using the RAunet model to extract the spatial distribution information of WW from GF-2 imagery, the experimental results showed that the mIou of the recognition results reached 92.48%, an improvement of 2.66%, 4.15%, 1.42%, 2.35%, 3.76%, and 0.47% compared to FCN, U-Net, DeepLabv3, SegNet, ResUNet, and UNet++, respectively. The superiority of the RAunet model in high-resolution images for WW extraction was verified in effectively improving the accuracy of the spatial distribution information extraction of WW.

We propose a remote sensing image semantic segmentation model based on dual attention and multi-scale feature fusion to solve the problems of objects scale differences and missing small objects. This model uses ResNet50 in the coding part to extract features. First of all, the output features of each stage of ResNet50 are introduced into the pyramid pooling module, making full use of the multi-scale context information of the image to cope with the change of the object scales. Secondly, the dual attention is introduced in the final output features of ResNet50 to establish the semantic relationship between the spatial and channel dimensions, which enhances the ability of feature representation and improve the condition that small targets are difficult to segment. Finally, starting from the output features of the attention module, the features of all levels are gradually integrated to complete decoding to refine the target segmentation edge. The designed comparative experiments results show the effectiveness of the proposed method.

Sea surface height data reconstruction via inter and intra layer features based on dual attention

Rapid and accurate detection of COVID-19 is a crucial step to control the virus. For this purpose, the authors designed a web-based COVID-19 detector using efficient dual attention networks, called ‘EDANet’. The EDANet architecture is based on inverted residual structures to reduce the model complexity and dual attention mechanism with position and channel attention blocks to enhance the discriminant features from the different layers of the network. Although the EDANet has only 4.1 million parameters, the experimental results demonstrate that it achieves the state-of-the-art results on the COVIDx data set in terms of accuracy and sensitivity of 96 and . The web application is available at the following link: https://covid19detector-cxr.herokuapp.com/.

Cross-modal retrieval aims to search samples of one modality via queries of other modalities, which is a hot issue in the community of multimedia. However, two main challenges, i.e., heterogeneity gap and semantic interaction across different modalities, have not been solved efficaciously. Reducing the heterogeneous gap can improve the cross-modal similarity measurement. Meanwhile, modeling cross-modal semantic interaction can capture the semantic correlations more accurately. To this end, this paper presents a novel end-to-end framework, called Dual Attention Generative Adversarial Network (DA-GAN). This technique is an adversarial semantic representation model with a dual attention mechanism, i.e., intra-modal attention and inter-modal attention. Intra-modal attention is used to focus on the important semantic feature within a modality, while inter-modal attention is to explore the semantic interaction between different modalities and then represent the high-level semantic correlation more precisely. A dual adversarial learning strategy is designed to generate modality-invariant representations, which can reduce the cross-modal heterogeneity efficiently. The experiments on three commonly used benchmarks show the better performance of DA-GAN than these competitors.

DAJLENet: A neural network based on dual attention and joint learning for explainable heart failure adverse event prediction