CRTransSar: A Visual Transformer Based on Contextual Joint Representation Learning for SAR Ship Detection

The current Synthetic Aperture Radar (SAR) target detection methods based on Convolutional Neural Network (CNN) rely on a large amount of slice-level labeled train samples. However, it takes a lot of labor and material resources to label the SAR images at slice-level. Compared to label samples at slice-level, it is easier to label them at image-level. The image-level label indicates whether the image contains the target of interest or not. In this paper, a semi-supervised SAR image target detection method based on CNN is proposed by using a small number of slice-level labeled samples and a large number of image-level labeled samples. The target detection network of this method consists of region proposal network and detection network. Firstly, the target detection network is trained using the slice-level labeled samples. After training convergence, the output slices constitute the candidate region set. Then, the image-level labeled clutter samples are input into the network and then the negative slices of the output are added to the candidate region set. Next, the image-level labeled target samples are input into the network as well. After selecting the positive and negative slices in the output of the network, they are added to the candidate region set. Finally, the detection network is trained using the updated candidate region set. The processes of updating candidate region set and training detection network alternate until convergence. The experimental results based on the measured data demonstrate that the performance of the proposed method is similar to the fully supervised training method using a much larger set of slice-level samples.

This paper studies the polarimetric synthetic aperture radar (SAR) image interpretation and target detection. The polarimetric target decomposition theorem and dominant scattering mechanism identification approach have been used widely in polarimetric SAR image interpretation. In order to get a good performance, we present a three-step image interpretation approach which involves speckle reduction and image classification, as pre-processing before the classical interpretation approach. Furthermore, we study the target detection based on the covariance matrix decomposition, and an odd-scattering detector and an even-scattering detector are formed and analyzed. Experimental results are given to verify the effectiveness of the image interpretation approach and to quantify the presented polarimetric detectors.

The existing Synthetic Aperture Radar (SAR) image target detection methods based on convolutional neural networks (CNNs) have achieved remarkable performance, but these methods require a large number of target-level labeled training samples to train the network. Moreover, some clutter is very similar to targets in SAR images with complex scenes, making the target detection task very difficult. Therefore, a SAR target detection network based on a semi-supervised learning and attention mechanism is proposed in this paper. Since the image-level label simply marks whether the image contains the target of interest or not, which is easier to be labeled than the target-level label, the proposed method uses a small number of target-level labeled training samples and a large number of image-level labeled training samples to train the network with a semi-supervised learning algorithm. The proposed network consists of a detection branch and a scene recognition branch with a feature extraction module and an attention module shared between these two branches. The feature extraction module can extract the deep features of the input SAR images, and the attention module can guide the network to focus on the target of interest while suppressing the clutter. During the semi-supervised learning process, the target-level labeled training samples will pass through the detection branch, while the image-level labeled training samples will pass through the scene recognition branch. During the test process, considering the help of global scene information in SAR images for detection, a novel coarse-to-fine detection procedure is proposed. After the coarse scene recognition determining whether the input SAR image contains the target of interest or not, the fine target detection is performed on the image that may contain the target. The experimental results based on the measured SAR dataset demonstrate that the proposed method can achieve better performance than the existing methods.

Synthetic Aperture Radar (SAR) images are prone to be contaminated by noise, which makes it very difficult to perform target recognition in SAR images. Inspired by great success of very deep convolutional neural networks (CNNs), this paper proposes a robust feature extraction method for SAR image target classification by adaptively fusing effective features from different CNN layers. First, YOLOv4 network is fine-tuned to detect the targets from the respective MF SAR target images. Second, a very deep CNN is trained from scratch on the moving and stationary target acquisition and recognition (MSTAR) database by using small filters throughout the whole net to reduce the speckle noise. Besides, using small-size convolution filters decreases the number of parameters in each layer and, therefore, reduces computation cost as the CNN goes deeper. The resulting CNN model is capable of extracting very deep features from the target images without performing any noise filtering or pre-processing techniques. Third, our approach proposes to use the multi-canonical correlation analysis (MCCA) to adaptively learn CNN features from different layers such that the resulting representations are highly linearly correlated and therefore can achieve better classification accuracy even if a simple linear support vector machine is used. Experimental results on the MSTAR dataset demonstrate that the proposed method outperforms the state-of-the-art methods.

In recent years, convolutional neural network (CNN)-based methods have been extensively explored for synthetic aperture radar (SAR) target detection. Nevertheless, the convolutional sampling locations of CNNs cannot accurately fit vehicle targets due to the fixed sampling mechanism in the convolutional kernel. In this paper, we focus on the vehicle target detection task in SAR images and propose a novel rectangle-invariant rotatable convolution (RIRConv) to determine more accurately the convolutional sampling locations for vehicle targets. Specifically, this paper considers the shape characteristic of vehicle targets in SAR images, which always retain a rectangular shape despite having varying sizes, aspect ratios, and rotation angles. The proposed RIRConv equips three additional learnable attribute parameters, namely, width, height, and angle attributes, to adaptively adjust the sampling locations in the convolutional kernel according to the targets. In addition, the RIRConv applies a modulation mechanism to focus on the sampling locations that significantly affect the output. Finally, the RIRConv is introduced into the single-shot multibox detector (SSD) to realize SAR vehicle target detection. In this way, the feature representation capability of SSD for vehicle targets can be enhanced, thus leading to higher detection performance. Notably, the proposed RIRConv is “plug-and-play” and can also be used with other existing advanced technologies to achieve higher detection performance. The experiments based on the measured miniSAR data validate the effectiveness of the proposed method.

Synthetic Aperture Radar (SAR) target classification is an important branch of SAR image interpretation. The deep learning based SAR target classification algorithms have made remarkable achievements. But the acquisition and annotation of SAR target images are time-consuming and laborious, and it is difficult to obtain sufficient training data in many cases. The insufficient training data can make deep learning based models suffering from over-fitting, which will severely limit their wide application in SAR target classification. Motivated by the above problem, this paper employs transfer-learning to transfer the prior knowledge learned from a simulated SAR dataset to a real SAR dataset. To overcome the sample restriction problem caused by the poor feature discriminability for real SAR data. A simple and effective sample spectral regularization method is proposed, which can regularize the singular values of each SAR image feature to improve the feature discriminability. Based on the proposed regularization method, we design a transfer-learning pipeline to leverage the simulated SAR data as well as acquire better feature discriminability. The experimental results indicate that the proposed method is feasible for the sample restriction problem in SAR target classification. Furthermore, the proposed method can improve the classification accuracy when relatively sufficient training data is available, and it can be plugged into any convolutional neural network (CNN) based SAR classification models.

Deep convolutional neural networks (DCNNs) have attracted extensive attention in synthetic aperture radar (SAR) image target detection. However, there is usually a lack of labeled data in SAR images which results in difficulties to improve the detection performance with DCNN based methods. Transfer learning borrows generic knowledge from other tasks for the target task and can be an effective solution to the small sample problem, but the features' generality and specificity across various models are not discussed in target detection. In addition, conventional DCNN based models have too many parameters related to the prior information of the targets which increase the difficulty of transfer learning between different tasks. In this paper, we mainly conduct research in three folds to address these mentioned issues. First, the generality and specificity of features in different network models are analyzed in the field of SAR target detection. Second, a transitive transfer framework is developed for detection under small sample condition. Third, an anchor free rotatable detector with flexible structure is designed for SAR images detection to suit the transfer detection framework proposed in this paper and a reasonable explanation is given for its upper hand with small samples. Experimental results demonstrate that: 1) transitive transfer framework is valid in SAR target detection; 2) the anchor free detector in this framework is superior to the anchor based one; 3) the proposed detector achieves better performance under small sample condition by comparison with state-of-the-art methods.

Synthetic aperture radar (SAR) image target detection and recognition (SAR-TDR) tasks have become research hot spots in the remote sensing application. These targets include ships, vehicles, aircraft, oil tanks, bridges, and so on. However, with the rapid development of SAR technology and increasingly complex electromagnetic environment, complex characteristics of SAR images bring severe challenges to the accurate SAR-TDR via traditional physical models or manual feature-extraction-based machine learning methods. In recent years, deep learning (DL), as a powerful automatic feature extraction algorithm, has been widely used in the computer vision domain. More specifically, DL has also been introduced into the SAR-TDR tasks and has effectively achieved good performance in terms of accuracy, real-time processing, etc. With the rapid development of DL, SAR image processing, and practical requirements of SAR-TDR in civilian and military domains, it is crucial to conduct a systematic survey on SAR-TDR in the past few years. In this survey article, we mainly conduct a systematic overview of DL-based SAR-TDR literature on two tasks, i.e., target recognition (e.g., ground vehicles, ships, and aircraft) and target detection (e.g., ships, aircraft, change detection, sea surface oil spills, and oil tanks). More specifically, our related works about these topics are also presented to verify the effectiveness of DL-based methods. First, several DL methods (e.g., convolutional neural networks, recurrent neural networks, autoencoders, and generative adversarial networks), commonly used in SAR-TDR, are briefly introduced. Then, a systematic review of DL-based SAR-TDR (including our related works) is presented. Finally, the current challenges and future possible research directions are deeply analyzed and discussed.

This tutorial presents a state-of-the-art survey of advanced applications of Synthetic Aperture Radar (SAR). Applications include: the detection and recognition of targets in SAR imagery, and the detection of targets under trees (and in the open) using change detection (coherent change detection, full-polarization change detection, etc). A fully polarimetric SAR sensor is described. Models of clutter and targets are developed from analysis of full-polarization SAR imagery. Algorithms for optimum (GLRT) processing of full-polarization data are derived, including the polarimetric whitening filter (PWF) and the polarimetric matched filter (PMF). Polarization-based discrimination features (polarization entropy, anisotropy, etc.) are developed and applied to SAR image terrain classification. Model-based and template-based target recognition systems are described and classifier performance results versus resolution and polarization are summarized for 10-and 20-target classifiers. Super-resolution algorithms are applied to SAR imagery and target recognition performance improvements demonstrated. Multi-polarization and multi-pass change detection algorithms are developed and demonstrated using UHF SAR imagery of targets under trees; fusion of multi-polarization coherent and non-coherent change detection algorithms is demonstrated. The effect of SAR image compression on change detection performance is quantified. Change detection performance using multi-pass VHF SAR imagery of targets under trees is demonstrated using data gathered by the Swedish CARABAS SAR system.

Ocean surveillance via high-resolution Synthetic Aperture Radar (SAR) imageries has been a hot issue because SAR is able to work in all-day and all-weather conditions. The launch of Chinese Gaofen-3 (GF-3) satellite has provided a large number of SAR imageries, making it possible to marine targets monitoring. However, it is difficult for traditional methods to extract effective features to classify and detect different types of marine targets in SAR images. This paper proposes a convolutional neutral network (CNN) model for marine target classification at patch level and an overall scheme for marine target detection in large-scale SAR images. First, eight types of marine targets in GF-3 SAR images are labelled based on feature analysis, building the datasets for further experiments. As for the classification task at patch level, a novel CNN model with six convolutional layers, three pooling layers, and two fully connected layers has been designed. With respect to the detection part, a Single Shot Multi-box Detector with a multi-resolution input (MR-SSD) is developed, which can extract more features at different resolution versions. In order to detect different targets in large-scale SAR images, a whole workflow including sea-land segmentation, cropping with overlapping, detection with MR-SSD model, coordinates mapping, and predicted boxes consolidation is developed. Experiments based on the GF-3 dataset demonstrate the merits of the proposed methods for marine target classification and detection.

The deep learning object detection algorithm has been widely applied in the field of synthetic aperture radar (SAR). By utilizing deep convolutional neural networks (CNNs) and other techniques, these algorithms can effectively identify and locate targets in SAR images, thereby improving the accuracy and efficiency of detection. In recent years, achieving real-time monitoring of regions has become a pressing need, leading to the direct completion of real-time SAR image target detection on airborne or satellite-borne real-time processing platforms. However, current GPU-based real-time processing platforms struggle to meet the power consumption requirements of airborne or satellite applications. To address this issue, a low-power, low-latency deep learning SAR object detection algorithm accelerator was designed in this study to enable real-time target detection on airborne and satellite SAR platforms. This accelerator proposes a Process Engine (PE) suitable for multidimensional convolution parallel computing, making full use of Field-Programmable Gate Array (FPGA) computing resources to reduce convolution computing time. Furthermore, a unique memory arrangement design based on this PE aims to enhance memory read/write efficiency while applying dataflow patterns suitable for FPGA computing to the accelerator to reduce computation latency. Our experimental results demonstrate that deploying the SAR object detection algorithm based on Yolov5s on this accelerator design, mounted on a Virtex 7 690t chip, consumes only 7 watts of dynamic power, achieving the capability to detect 52.19 512 × 512-sized SAR images per second.

Although Synthetic Aperture Radar (SAR) can capture rich land cover information as a most important advanced technique in the field of international earth observation, the application effects still limited significantly. The reason is that the study of SAR imaging processing, SAR image processing and SAR applications are conducted respectively, the integrated study are lacked for parameter selection of SAR imaging processing and SAR image processing and targets identification of SAR application. Focusing on above science problems, the study for phase high fidelity model of high precision interferometric SAR, SAR non-stationary backscattering characteristics and cognition, SAR three-dimensional electromagnetic scattering modeling and its scattering mechanism and SAR multi-parameters optimization information extraction has been conducted. The typical natural distributed targets and man-made targets such as surface deformation, sea ice, building complex and collapsed building are selected, the method and application study oriented to SAR environmental parameters inversion are conduced, the main result and conclusion are as follows: (1) The overall theoretical frames of SAR information integrated processing was proposed; The simulation and ground validation system used to assess the effects of SAR information integrated processing was created; furthermore, the spaceborne SAR imaging algorithm with three steps focusing processing was proposed and realized the multi-modal integrated imaging processing and improved the imaging processing precision. (2) In terms of SAR information integrated processing of natural targets, focusing on phase high fidelity imaging processing of interferometric SAR surface deformation monitoring, an novel spaceborne SAR high fidelity simulation method based on stationary RCS and improved Goldstein SAR interferogram filter based on empirical mode decomposition were proposed. Focusing on multi-parameters optimization processing of sea ice types classification, the multi-channel spaceborne SAR sparse imaging method based on compressive sensing, Kalman filter for removal of scalloping and inter-scan banding in ScanSAR images and SVM sea ice classification method combined with sea ice concentration were proposed. (3) In terms of SAR information integrated processing of man-made targets, focusing on non-stationary backscattering from building complex, mitigation of azimuth ambiguities in spaceborne stripmap SAR iamges using selective restoration, SAR image despeckling by selective 3D filtering of multiple compressive reconstructed images and man-made target detection in urban areas based on a new azimuth stationary extraction method were proposed. Focusing on three-dimensional modeling and fast extraction of collapsed building, H-a-p method used for collapsed building extraction was proposed.

The local structure feature of the target in synthetic aperture radar (SAR) image and the inner correlation among multiple SAR images of the same target can effectively improve the recognition performance. In order to utilize these two kinds of information, we propose the multi-view tensor sparse representation (MTSR) model for SAR target recognition. In the proposed model, the SAR image is treated as tensor and represented by the dictionaries on each dimension of the image and corresponding sparse representation coefficient tensor (SRCT). For multi-view SAR images of one target, we design the proposed model by letting the SRCTs of those images have the same structure, which means that the non-zero elements of those SRCTs have the same coordinates. In order to solve the proposed model, we propose the joint tensor orthogonal matching pursuit (JT-OMP) algorithm to calculate the SRCTs of multiple views. JT-OMP ensures that all the SRCTs have the same structure by looking for the atoms of dictionaries that can contribute the total maximum energy for all multi-view tensors in every iteration. To achieve recognition, we construct two dictionaries for each class and compare the total sparse representation error of multi-view SAR images in each class. The experiments conducted on the moving and stationary target acquisition recognition database (MSTAR) verify the performance of the proposed algorithm.

In recent years, the rapid development of Deep Learning (DL) has provided a new method for ship detection in Synthetic Aperture Radar (SAR) images. However, there are still four challenges in this task. (1) The ship targets in SAR images are very sparse. A large number of unnecessary anchor boxes may be generated on the feature map when using traditional anchor-based detection models, which could greatly increase the amount of computation and make it difficult to achieve real-time rapid detection. (2) The size of the ship targets in SAR images is relatively small. Most of the detection methods have poor performance on small ships in large scenes. (3) The terrestrial background in SAR images is very complicated. Ship targets are susceptible to interference from complex backgrounds, and there are serious false detections and missed detections. (4) The ship targets in SAR images are characterized by a large aspect ratio, arbitrary direction and dense arrangement. Traditional horizontal box detection can cause non-target areas to interfere with the extraction of ship features, and it is difficult to accurately express the length, width and axial information of ship targets. To solve these problems, we propose an effective lightweight anchor-free detector called R-Centernet+ in the paper. Its features are as follows: the Convolutional Block Attention Module (CBAM) is introduced to the backbone network to improve the focusing ability on small ships; the Foreground Enhance Module (FEM) is used to introduce foreground information to reduce the interference of the complex background; the detection head that can output the ship angle map is designed to realize the rotation detection of ship targets. To verify the validity of the proposed model in this paper, experiments are performed on two public SAR image datasets, i.e., SAR Ship Detection Dataset (SSDD) and AIR-SARShip. The results show that the proposed R-Centernet+ detector can detect both inshore and offshore ships with higher accuracy than traditional models with an average precision of 95.11% on SSDD and 84.89% on AIR-SARShip, and the detection speed is quite fast with 33 frames per second.

As an active microwave imaging sensor, Synthetic Aperture Radar (SAR) has become one of the main means of Earth observation owing to its unique technical advantages of all-day, all-weather operation and long working distance. As such, it plays a very important role in military and civilian fields. With the development of SAR remote-sensing technology, high-resolution, high-quality SAR images are produced continuously. However, manual detection and recognition of targets of interest is time-consuming and laborious, so the development of Automatic Target Recognition (ATR) technology is a matter of urgency. The typical SAR ATR system primarily comprises three stages: detection, discrimination, and classification/recognition. The detection and discrimination stages are the basis of the SAR ATR system, and research on SAR applications in the radar field has been conducted by researchers around the world. For single-channel SAR images, target detection and discrimination from simple scenes yield good results. However, in complex scenes, the clutter scattering intensity is relatively high, the clutter background is heterogenous, the target scattering intensity is relatively weak, and the target distribution is dense. These factors continue to make accurate SAR target detection and discrimination difficult. In this paper, we summarize the recent research progress on single-channel SAR target detection and discrimination methods for complex scenes, analyze the characteristics and problems associated with various methods, and consider the future development trend of single-channel SAR target detection and discrimination methods for complex scenes.