An Improved SAR Image Semantic Segmentation Deeplabv3+ Network Based on the Feature Post-Processing Module

In Earth observation, cloud severely affects the interpretation of optical satellites generated high-resolution images. Cloud-free optical images are vital for downstream tasks such as semantic segmentation and object detection. Thus, the elimination of clouds from optical imagery has emerged as a significant topic in remote sensing. Currently, most existing methods are proposed to leverage the texture information from auxiliary synthetic aperture radar (SAR) images to restore cloud-free images via direct channel merging. However, such a unified feature extraction approach often neglects the inherent distribution disparity between SAR and optical images—the result of differing imaging principles-potentially leading to significant feature loss. To this end, we introduce a network by jointing SAR and optical images multimodal and cross-attention network (MMCANet) to effectively extract multiscale contextual features from SAR imagery and integrate them with optical features. Specifically, instead of simple concatenation of the channels of SAR and optical images, we obtain high-dimensional features from them through independent feature extractors. The integration of these features is facilitated by a cross-attention mechanism that provides a more fine-grained amalgamation of information. Meanwhile, an atrous spatial pyramid pooling (ASPP) module is introduced into the integration of high-level features, which captures multiscale contextual information around clouded areas. In addition, we propose four advanced remote sensing image restoration algorithms that approach image restoration as a series of subtasks, gradually eliminating clouds to enhance performance. Comprehensive assessments show that MMCANet performs well on the SEN 12 MS-CR dataset with peak signal-to-noise ratio (PSNR) of 39.8871, structural similarity index (SSIM) of 0.9672, mean absolute error (MAE) of 0.0081, and spectral angle mapper (SAM) of 2.9884.

In recent years, the rapid development of Deep Learning (DL) has provided a new method for ship detection in Synthetic Aperture Radar (SAR) images. However, there are still four challenges in this task. (1) The ship targets in SAR images are very sparse. A large number of unnecessary anchor boxes may be generated on the feature map when using traditional anchor-based detection models, which could greatly increase the amount of computation and make it difficult to achieve real-time rapid detection. (2) The size of the ship targets in SAR images is relatively small. Most of the detection methods have poor performance on small ships in large scenes. (3) The terrestrial background in SAR images is very complicated. Ship targets are susceptible to interference from complex backgrounds, and there are serious false detections and missed detections. (4) The ship targets in SAR images are characterized by a large aspect ratio, arbitrary direction and dense arrangement. Traditional horizontal box detection can cause non-target areas to interfere with the extraction of ship features, and it is difficult to accurately express the length, width and axial information of ship targets. To solve these problems, we propose an effective lightweight anchor-free detector called R-Centernet+ in the paper. Its features are as follows: the Convolutional Block Attention Module (CBAM) is introduced to the backbone network to improve the focusing ability on small ships; the Foreground Enhance Module (FEM) is used to introduce foreground information to reduce the interference of the complex background; the detection head that can output the ship angle map is designed to realize the rotation detection of ship targets. To verify the validity of the proposed model in this paper, experiments are performed on two public SAR image datasets, i.e., SAR Ship Detection Dataset (SSDD) and AIR-SARShip. The results show that the proposed R-Centernet+ detector can detect both inshore and offshore ships with higher accuracy than traditional models with an average precision of 95.11% on SSDD and 84.89% on AIR-SARShip, and the detection speed is quite fast with 33 frames per second.

In light of recent advances in deep learning and Synthetic Aperture Radar (SAR) technology, there has been a growing adoption of ship detection models that are based on deep learning methodologies. However, the efficiency of SAR ship detection models is significantly impacted by complex backgrounds, noise, and multi-scale ships (the number of pixels occupied by ships in SAR images varies significantly). To address the aforementioned issues, this research proposes a Pyramid Pooling Attention Network (PPA-Net) for SAR multi-scale ship detection. Firstly, a Pyramid Pooled Attention Module (PPAM) is designed to alleviate the influence of background noise on ship detection while its parallel component favors the processing of multiple ship sizes. Different from the previous attention module, the PPAM module can better suppress the background noise in SAR images because it considers the saliency of ships in SAR images. Secondly, an Adaptive Feature Balancing Module (AFBM) is developed, which can automatically balance the conflict between ship semantic information and location information. Finally, the detection capabilities of the ship detection model for multi-scale ships are further improved by introducing the Atrous Spatial Pyramid Pooling (ASPP) module. This innovative module enhances the detection model’s ability to detect ships of varying scales by extracting features from multiple scales using atrous convolutions and spatial pyramid pooling. PPA-Net achieved detection accuracies of 95.19% and 89.27% on the High-Resolution SAR Images Dataset (HRSID) and the SAR Ship Detection Dataset (SSDD), respectively. The experimental results demonstrate that PPA-Net outperforms other ship detection models.

Ocean surveillance via high-resolution Synthetic Aperture Radar (SAR) imageries has been a hot issue because SAR is able to work in all-day and all-weather conditions. The launch of Chinese Gaofen-3 (GF-3) satellite has provided a large number of SAR imageries, making it possible to marine targets monitoring. However, it is difficult for traditional methods to extract effective features to classify and detect different types of marine targets in SAR images. This paper proposes a convolutional neutral network (CNN) model for marine target classification at patch level and an overall scheme for marine target detection in large-scale SAR images. First, eight types of marine targets in GF-3 SAR images are labelled based on feature analysis, building the datasets for further experiments. As for the classification task at patch level, a novel CNN model with six convolutional layers, three pooling layers, and two fully connected layers has been designed. With respect to the detection part, a Single Shot Multi-box Detector with a multi-resolution input (MR-SSD) is developed, which can extract more features at different resolution versions. In order to detect different targets in large-scale SAR images, a whole workflow including sea-land segmentation, cropping with overlapping, detection with MR-SSD model, coordinates mapping, and predicted boxes consolidation is developed. Experiments based on the GF-3 dataset demonstrate the merits of the proposed methods for marine target classification and detection.

This study aims to enhance the efficiency and accuracy of thyroid nodule segmentation in ultrasound images, ultimately improving nodule detection and diagnosis. For clinical deployment on mobile and embedded devices, DeepLabV3+ strives to achieve a balance between a lightweight architecture and high segmentation accuracy. A comprehensive dataset of ultrasound images was meticulously curated using a high-resolution ultrasound imaging device. Data acquisition adhered to standardized protocols to ensure high-quality imaging. Preprocessing steps, including noise reduction and contrast optimization, were applied to enhance image clarity. Expert radiologists provided ground truth labels through meticulous annotation. To improve segmentation performance, we integrated MobileNetV2 and Depthwise Separable Dilated Convolution into the Atrous Spatial Pyramid Pooling (ASPP) module, incorporating the Pyramid Pooling Module (PPM) and attention mechanisms. To mitigate classification imbalances, we employed Tversky loss functions in the ultrasound image classification process. In semantic image segmentation, DeepLabV3+ achieved an impressive Intersection over Union (IoU) of 94.37%, while utilizing only 12.4 MB of parameters, including weights and biases. This remarkable accuracy demonstrates the effectiveness of our approach. A high IoU value in medical imaging analysis reflects the model's ability to accurately delineate object boundaries. DeepLabV3+ represents a significant advancement in thyroid nodule segmentation, particularly for thyroid cancer screening and diagnosis. The obtained segmentation results suggest promising directions for future research, especially in the early detection of thyroid nodules. Deploying this algorithm on mobile devices offers a practical solution for early diagnosis and is likely to improve patient outcomes.

This paper proposes an Synthetic Aperture Radar (SAR) imaging and detection model of multiple targets at the maritime scene. The sea surface sample is generated according to the composite rough surface theory. The SAR imaging model is constructed based on a hybrid EM calculation approach with the fast ray tracing strategy and the modified facet Small Slope Approximation (SSA) solution. Numerical simulations calculate the EM scattering and the SAR imaging of the multiple cone targets above the sea surface, with the scattering mechanisms analyzed and discussed. The SAR imaging datasets are then set up by the SAR image simulations. A modified YOLOv7 neural network with the Spatial Pyramid Pooling Fast Connected Spatial Pyramid Convolution (SPPFCSPC) module, Convolutional Block Attention Module (CBAM), modified Feature Pyramid Network (FPN) structure and extra detection head is developed. In the training process on our constructed SAR datasets, the precision rate, recall rate, mAP@0.5 and mAP@0.5:0.95 are 97.46%, 90.08%, 92.91% and 91.98%, respectively, after 300 rounds of training. The detection results show that the modified YOLOv7 has a good performance in selecting the targets out of the complex sea surface and multipath interference background.

The existing Synthetic Aperture Radar (SAR) image target detection methods based on convolutional neural networks (CNNs) have achieved remarkable performance, but these methods require a large number of target-level labeled training samples to train the network. Moreover, some clutter is very similar to targets in SAR images with complex scenes, making the target detection task very difficult. Therefore, a SAR target detection network based on a semi-supervised learning and attention mechanism is proposed in this paper. Since the image-level label simply marks whether the image contains the target of interest or not, which is easier to be labeled than the target-level label, the proposed method uses a small number of target-level labeled training samples and a large number of image-level labeled training samples to train the network with a semi-supervised learning algorithm. The proposed network consists of a detection branch and a scene recognition branch with a feature extraction module and an attention module shared between these two branches. The feature extraction module can extract the deep features of the input SAR images, and the attention module can guide the network to focus on the target of interest while suppressing the clutter. During the semi-supervised learning process, the target-level labeled training samples will pass through the detection branch, while the image-level labeled training samples will pass through the scene recognition branch. During the test process, considering the help of global scene information in SAR images for detection, a novel coarse-to-fine detection procedure is proposed. After the coarse scene recognition determining whether the input SAR image contains the target of interest or not, the fine target detection is performed on the image that may contain the target. The experimental results based on the measured SAR dataset demonstrate that the proposed method can achieve better performance than the existing methods.

Monitoring aircraft using synthetic aperture radar (SAR) images is a very important task. Given its coherent imaging characteristics, there is a large amount of speckle interference in the image. This phenomenon leads to the scattering information of aircraft targets being masked in SAR images, which is easily confused with background scattering points. Therefore, automatic detection of aircraft targets in SAR images remains a challenging task. For this task, this paper proposes a framework for speckle reduction preprocessing of SAR images, followed by the use of an improved deep learning method to detect aircraft in SAR images. Firstly, to improve the problem of introducing artifacts or excessive smoothing in speckle reduction using total variation (TV) methods, this paper proposes a new nonconvex total variation (NTV) method. This method aims to ensure the effectiveness of speckle reduction while preserving the original scattering information as much as possible. Next, we present a framework for aircraft detection based on You Only Look Once v8 (YOLOv8) for SAR images. Therefore, the complete framework is called SAR-NTV-YOLOv8. Meanwhile, a high-resolution small target feature head is proposed to mitigate the impact of scale changes and loss of depth feature details on detection accuracy. Then, an efficient multi-scale attention module was proposed, aimed at effectively establishing short-term and long-term dependencies between feature grouping and multi-scale structures. In addition, the progressive feature pyramid network was chosen to avoid information loss or degradation in multi-level transmission during the bottom-up feature extraction process in Backbone. Sufficient comparative experiments, speckle reduction experiments, and ablation experiments are conducted on the SAR-Aircraft-1.0 and SADD datasets. The results have demonstrated the effectiveness of SAR-NTV-YOLOv8, which has the most advanced performance compared to other mainstream algorithms.

Synthetic aperture radar (SAR) image target recognition technology is aimed at automatically determining the presence or absence of target information from the input SAR image and improving the efficiency and accuracy of SAR image interpretation. Based on big data analysis, dirty data is removed, clean data is returned, and standardized processing of SAR image data is realized. At the same time, by establishing a statistical model of coherent speckles, the convolutional autoencoder is used to denoise the SAR image. Finally, the network model modified by softmax cross‐entropy loss and Fisher loss is used for automatic target recognition. Based on the MSTAR data set, two scene graphs containing the target synthesized by the background image and the target slice are used for experiments. Several comparative experiments have verified the effectiveness of the classification and recognition model in this paper.

Synthetic aperture radar (SAR) image target detection and recognition (SAR-TDR) tasks have become research hot spots in the remote sensing application. These targets include ships, vehicles, aircraft, oil tanks, bridges, and so on. However, with the rapid development of SAR technology and increasingly complex electromagnetic environment, complex characteristics of SAR images bring severe challenges to the accurate SAR-TDR via traditional physical models or manual feature-extraction-based machine learning methods. In recent years, deep learning (DL), as a powerful automatic feature extraction algorithm, has been widely used in the computer vision domain. More specifically, DL has also been introduced into the SAR-TDR tasks and has effectively achieved good performance in terms of accuracy, real-time processing, etc. With the rapid development of DL, SAR image processing, and practical requirements of SAR-TDR in civilian and military domains, it is crucial to conduct a systematic survey on SAR-TDR in the past few years. In this survey article, we mainly conduct a systematic overview of DL-based SAR-TDR literature on two tasks, i.e., target recognition (e.g., ground vehicles, ships, and aircraft) and target detection (e.g., ships, aircraft, change detection, sea surface oil spills, and oil tanks). More specifically, our related works about these topics are also presented to verify the effectiveness of DL-based methods. First, several DL methods (e.g., convolutional neural networks, recurrent neural networks, autoencoders, and generative adversarial networks), commonly used in SAR-TDR, are briefly introduced. Then, a systematic review of DL-based SAR-TDR (including our related works) is presented. Finally, the current challenges and future possible research directions are deeply analyzed and discussed.

Fast and reliable target recognition of the synthetic aperture radar (SAR) images has been widely used in the fields of the marine monitoring, military reconnaissance and strike all over the world. However, due to the difficulty of the intra-class difference and inter-class similarity of the multiclass targets in the high resolution SAR images, the existing methods are difficult to recognize the targets accurately when facing the spaceborne platforms with the high resource constraints. Therefore, in order to solve the above problems, we propose a novel recognition method based on the convolutional neural network (CNN). Firstly, we propose a lightweight CNN framework which regards densely connected convolutional network (DenseNet) as the baseline. Secondly, we advocate a strong discriminative loss function which efficiently improves the recognition accuracy of the targets in the spaceborne SAR images. Experiments are conducted on the TerraSAR dataset and MSTAR dataset to evaluate the proposed method. The results show that our method performs better than the baseline on the both benchmark datasets.

In this article, an application of clutter modeling and reduction techniques to synthetic aperture radar (SAR) images of moving and stationary target acquisition and recognition data is presented. Statistical modeling of the clutter signal within these particular SAR images is demonstrated. Lognormal, Weibull, and K‐distribution models are analyzed for the amplitude distribution of high‐resolution land clutter data. Higher‐order statistics (moments and cumulants) are utilized to estimate the appropriate statistical distribution models for the clutter. Also, Kolmogorov‐Smirnov (K‐S) goodness‐of‐fit test is employed to validate the accuracy of the selected models. With the use of the determined clutter model, constant false‐alarm rate detection algorithm is applied to the SAR images of several military targets. Resultant SAR images obtained by using the proposed method show that target signatures are reliably differentiated from the clutter background. © 2008 Wiley Periodicals, Inc. Microwave Opt Technol Lett 50: 1514–1520, 2008; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/mop.23413

Although vehicles traveling on the sea or roads routinely experience acceleration, the parameter estimation algorithms in synthetic aperture radar (SAR) imaging assume that the target velocity stays the same, which will lead to erroneous estimates of the motion parameters. In this letter, the estimation of the velocity and acceleration of the moving targets in SAR imaging are investigated at the same time. First, a three-order range equation with the velocity and acceleration components of the target is established. Then an algorithm based on cubic phase function (CPF) is presented to estimate the so called Doppler phase coefficient (DPC) of the multi-static SAR azimuth signals, and the velocity and acceleration of the moving targets can be estimated on the basis of the relationship between the DPC and motion parameters ultimately. Numerical results show that the method is effective.

In recent years, convolutional neural network (CNN)-based methods have been extensively explored for synthetic aperture radar (SAR) target detection. Nevertheless, the convolutional sampling locations of CNNs cannot accurately fit vehicle targets due to the fixed sampling mechanism in the convolutional kernel. In this paper, we focus on the vehicle target detection task in SAR images and propose a novel rectangle-invariant rotatable convolution (RIRConv) to determine more accurately the convolutional sampling locations for vehicle targets. Specifically, this paper considers the shape characteristic of vehicle targets in SAR images, which always retain a rectangular shape despite having varying sizes, aspect ratios, and rotation angles. The proposed RIRConv equips three additional learnable attribute parameters, namely, width, height, and angle attributes, to adaptively adjust the sampling locations in the convolutional kernel according to the targets. In addition, the RIRConv applies a modulation mechanism to focus on the sampling locations that significantly affect the output. Finally, the RIRConv is introduced into the single-shot multibox detector (SSD) to realize SAR vehicle target detection. In this way, the feature representation capability of SSD for vehicle targets can be enhanced, thus leading to higher detection performance. Notably, the proposed RIRConv is “plug-and-play” and can also be used with other existing advanced technologies to achieve higher detection performance. The experiments based on the measured miniSAR data validate the effectiveness of the proposed method.

The deep learning object detection algorithm has been widely applied in the field of synthetic aperture radar (SAR). By utilizing deep convolutional neural networks (CNNs) and other techniques, these algorithms can effectively identify and locate targets in SAR images, thereby improving the accuracy and efficiency of detection. In recent years, achieving real-time monitoring of regions has become a pressing need, leading to the direct completion of real-time SAR image target detection on airborne or satellite-borne real-time processing platforms. However, current GPU-based real-time processing platforms struggle to meet the power consumption requirements of airborne or satellite applications. To address this issue, a low-power, low-latency deep learning SAR object detection algorithm accelerator was designed in this study to enable real-time target detection on airborne and satellite SAR platforms. This accelerator proposes a Process Engine (PE) suitable for multidimensional convolution parallel computing, making full use of Field-Programmable Gate Array (FPGA) computing resources to reduce convolution computing time. Furthermore, a unique memory arrangement design based on this PE aims to enhance memory read/write efficiency while applying dataflow patterns suitable for FPGA computing to the accelerator to reduce computation latency. Our experimental results demonstrate that deploying the SAR object detection algorithm based on Yolov5s on this accelerator design, mounted on a Virtex 7 690t chip, consumes only 7 watts of dynamic power, achieving the capability to detect 52.19 512 × 512-sized SAR images per second.