Conditional Diffusion for SAR to Optical Image Translation

Ship detection from synthetic aperture radar (SAR) images is a hot topic, but the difficulty in collecting labeled SAR images may hinder the development of deep-learning-based detection methods. Inspired by the idea of domain adaptation, in this article, we propose a hierarchical similarity alignment neural network (HSANet) for ship detection in SAR images, which is a domain adaptive (DA) approach with optical remote sensing images as training samples. The kernel target of HSANet is to mine and align both the global structure and the local instance information between SAR and optical images, where two modules, structural alignment module (SAM) and prototype alignment module (PAM), are designed to, respectively, conduct two hierarchies of alignment process. In general, SAM attempts to extract the global structure similarity which exists in image-level feature representation, while PAM tends to extract the local shape similarity which is instance-level representation. To be specific, SAM is developed by Fourier-based feature alignment, which tries to describe the similar structural relationship between optical and SAR images. Meanwhile, PAM is proposed based on the conjoint confidence analysis where the instance-level ship representations of the source and target domains is aligned. SAM and PAM work together to construct a hierarchical domain adaptation network for SAR ship detection. Experiments on several public datasets may indicate the effectiveness of the proposed method.

Synthetic aperture radar (SAR) and optical imaging are different remote sensing methods. Given a SAR image, is it possible to predict what the observed scene looks like in an optical image? Transfer between SAR data and optical data seems to be impossible. However, this article shows examples that by applying deep learning techniques on high resolution airborne SAR images and GoogleEarth optical images, the SAR images and optical images can transfer with each other. The transferring help us to better understand the relationship between SAR and optical image, and can be potentially used to transfer detection or classification algorithms for optical image straightforwardly to be applied on SAR image.

Reliable and accurate land use/land cover (LULC) map is a crucial data source for the understanding of coupled human–environment systems, monitoring changes, timely low-cost planning, and management of natural resources. Improvements in sensor technologies and machine learning capabilities have shifted the attention of remote sensing community to data complementarity through fusion of multi-sensor data for accurate feature extraction and mapping. Amalgamation of optical and synthetic aperture radar (SAR) images has shown promising advantages in enhancing the accuracy of extracting LULC as such method allows exploitation of information in sensors. This study investigated the potential of using freely available multisource Sentinel images to extract LULC maps in semi-arid environments through multi-scale geographic object-based image analysis (GEOBIA). A multi-scale classification framework that integrates GEOBIA, correlation-based feature selection (CFS), and random forest (RF)-supervised classification was adopted to extract LULC from assimilation of Sentinel multi-sensor products. First, Sentinel-1 and -2 images were pre-processed. Second, optimum multi-scale segmentation levels were selected using F-score segmentation quality measures. Third, 70 features of various spectral indices and derivatives and geometrical features from optical data and multiple ratios and textural features from dual-polarization SAR images were computed, and a CFS based on wrapper approach was used to select the most significant features at multi-scale levels. Finally, a single and multi-scale RF classifier was used to extract LULC classes using the most relevant features extracted from Sentinel SAR and optical images. Results of multi-scale image segmentation optimization showed that scale parameter (SP) values of 40, 60, and 150 were optimal for extraction of LULC classes. Results of feature selection showed that 22, 24, and 27 features were selected at scale SP values of 40, 60, and 150, respectively. Half of the features were common among the three scales. Single RF classification yielded overall accuracy (OA) values of 92.10%, 93%, and 91% and kappa coefficients of 0.901, 0.912, and 0.89 at scale values of 150, 60, and 40, respectively. Multiscale RF classification from scale values of 150 and 60 produced better LULC classification with OA 96.06% and kappa coefficient of 0.95 compared with other scale SP values. The integrated approach demonstrated an effective and promising method for high-quality LULC extraction from coupling optical and SAR images. Overall, multi-sensor Sentinel images along with the adopted approach feature a remarkable potential for improving LULC extraction and can effectively be used to update geographic information system layers for various applications.

In the field of remote sensing, cloud cover severely reduces the quality of satellite observations of the earth. Due to the complete absence of information in cloud-covered regions, cloud removal with a single optical image is an ill-posed problem. Since the synthetic aperture radar (SAR) can effectively penetrate clouds, fusing SAR and optical remote sensing images will effectively alleviate this problem. However, existing SAR-based optical cloud removal methods fail to effectively leverage the global information provided by the SAR image, resulting in limited performance gains. In this paper, we introduce a novel cloud removal method named the Multi-Level SAR-Guided Contextual Attention Network (MSGCA-Net). MSGCA-Net is designed with a multi-level architecture that integrates a SAR-Guided Contextual Attention (SGCA) module to fuse the dependable global contextual information from SAR images with the local features of optical images effectively. In the module of SGCA, the SAR image provides reliable global contextual information and genuine structure of cloud-covered regions, while the optical image provides the local feature information. The proposed model can efficiently extract and fuse global and local contextual information in SAR and optical images. We trained and evaluated the performance of the model on both simulated and real-world datasets. Both qualitative and quantitative experimental evaluation demonstrated that the proposed method can yield high quality cloud-free images and outperform state-of-the-art cloud removal methods.

The usage of a hypersonic platform for remote sensing application has promising prospects, especially for hypersonic platform-borne synthetic aperture radar (SAR) imaging. However, the high-speed of hypersonic platform will lead to extreme friction between the platform and air, which will cause the ionization of air. The ionized gas forms the plasma sheath wrapped around the hypersonic platform. The plasma sheath will severely affect the propagation of SAR signal and further affect the SAR imaging. Therefore, hypersonic platform-borne SAR imaging should be studied from a physical perspective. In this paper, hypersonic platform-borne SAR imaging under plasma sheath is analyzed. The SAR signal propagation in plasma sheath is computed using scatter matrix method. The proposed SAR signal model is verified by using a ground experiment system. Moreover, the effect of attenuation caused by plasma sheath on SAR imaging is studied under different SAR parameters and plasma sheath. The result shows that attenuation caused by plasma sheath will degrade the SAR imaging quality and even cause the point and area targets to be submerged into the noise. The real SAR images under plasma sheath also illustrate this phenomenon. Furthermore, by studying imaging results under different SAR and plasma parameters, it can be concluded that the severe degradation of SAR imaging quality appears at condition of high plasma sheath electron density and low SAR carrier frequency. The work in this paper will be beneficial for the study of hypersonic platform-borne SAR imaging and design of hypersonic SAR imaging systems in the future.

Accurate optical and synthetic aperture radar (SAR) image registration is crucial to multisensor remote sensing applications. Though several algorithms have been proposed, the practical implementation that directly matches large and high-resolution optical and SAR images remains underexplored by the community. Equipped with the satellite positioning parameters, optical and SAR images can be roughly registered based on geographic coordinates. However, the relative positioning accuracy is still dozens, even hundreds of pixels due to the inaccuracies of sensor parameters and elevation. Consequently, we propose a robust registration algorithm, which consists of two stages, where the horizontal positioning errors can be reduced in the first stage, and then, we fine-tune the correspondences in the second stage. Specifically, we propose a novel template matching method based on the dilated convolutional feature (DCF) and epipolar-oriented phase correlation. DCF is constructed by a depthwise-separable dilated convolution with multichannel gradients, which are generated by the Sobel operator for optical images and a ratio of exponentially weighted averages (ROEWA) operator for SAR images. Due to the large reception field of dilated convolution, DCF can retain invariance even for large relative positioning errors. The epipolar-oriented rectangle template, which stretches along the epipolar line, is then proposed to capture more overlapping areas compared to square templates. Furthermore, the outlier removal is implemented in the coordinate system of optical images to avoid the effect of range compression in SAR images. Inliers are finally used to refine the rational polynomial coefficients (RPCs) based on the bundle adjustment technique. Experimental results on high-resolution optical and SAR image products of various scenarios demonstrate the effectiveness of the proposed registration framework. The relative positioning errors of the refined RPCs can be reduced from hundreds of pixels to the subpixel level.

Synthetic aperture radar (SAR) image super-resolution (SR) has broad application prospects in military reconnaissance, marine monitoring, crop monitoring, and other fields. However, when the scale factor is large, current SAR image SR has limitations, such as insufficient feature extraction, difficulty of feature representation, and modeling difficulty of SR reconstruction. These problems have severely restricted the SR performance of SAR images. In addition, optical remote sensing images share higher resolution and richer high-frequency information than SAR remote sensing images. Therefore, we adopt high-resolution (HR) optical images as auxiliary images of SAR image SR to improve the performance of SAR image SR. Our research is conducted from two aspects: the extraction efficiency of HR optical image-guided information and the representation ability of low-resolution (LR) SAR image features. We propose an optical guidance residual networks (OGRNs) based on residual learning for SAR image SR. The OGRN uses optical images during network training, which enhances the performance of SAR image SR, even when optical images are missing during network testing. First, we extract the texture feature of SAR images through the designed residual learning module. Second, we extract the weight map of feature information for SAR image SR from HR optical images by the convolutional layer. Finally, the weight map is used to guide the weighted supervised SR reconstruction for SAR images. Extensive experimental results demonstrated that, under the guidance of optical images, OGRN achieves excellent performance in terms of both quantitative assessment metrics and visual quality.

In general, synthetic aperture radar (SAR) imaging and image processing are two sequential steps in SAR image processing. Due to the large size of SAR images, most image processing algorithms require image segmentation before processing. However, the existence of speckle noise in SAR images, as well as poor contrast and the uneven distribution of gray values in the same target, make SAR images difficult to segment. In order to facilitate the subsequent processing of SAR images, this paper proposes a new method that combines the back-projection algorithm (BPA) and a first-order gradient operator to enhance the edges of SAR images to overcome image segmentation problems. For complex-valued signals, the gradient operator was applied directly to the imaging process. The experimental results of simulated images and real images validate our proposed method. For the simulated scene, the supervised image segmentation evaluation indexes of our method have more than 1.18%, 11.2% and 11.72% improvement on probabilistic Rand index (PRI), variability index (VI), and global consistency error (GCE). The proposed imaging method will make SAR image segmentation and related applications easier.

This paper proposes a novel method on Synthetic Aperture Radar (SAR) Image denoising, which is based on context modeling combined with multiscale orthogonal bandlet coefficients. For the main influence of SAR image is multiplicative speckle noise, the logarithm uniform transform is used to convert it to additive noise. In this paper, the multiscale orthogonal bandlet transform is applied to noisy SAR Image. The contextual model is used to compute contextual values of bandlet coefficients on each scale. Therefore, bandlet coefficients are reconstructed into some parts by contextual values. The spatially adaptive soft-threshold (T) is to denoise SAR image in each part. Experimental results demonstrate that this method outperforms the methods of Ehance-Lee (E-lee), contextual model based on wavelet domain (Wavelet-context), and Hidden Markov Tree model bases on wavelet domain (Wavelet-hmt) on there indexes: Floating index (FI), Standard deviation and Edge saving index of vertical direction, by using a real SAR image. However, the method can not only reduce the speckle noise but also preserve edges and high frequency information, and gets considerable visual effect on SAR Image denoising.

Thanks to the availability of large-scale data, deep Convolutional Neural Networks (CNNs) have witnessed success in various applications of computer vision. However, the performance of CNNs on Synthetic Aperture Radar (SAR) image classification is unsatisfactory due to the lack of well-labeled SAR data, as well as the differences in imaging mechanisms between SAR images and optical images. Therefore, this paper addresses the problem of SAR image classification by employing the Generative Adversarial Network (GAN) to produce more labeled SAR data. We propose special GANs for generating SAR images to be used in the training process. First, we incorporate the quadratic operation into the GAN, extending the convolution to make the discriminator better represent the SAR data; second, the statistical characteristics of SAR images are integrated into the GAN to make its value function more reasonable; finally, two types of parallel connected GANs are designed, one of which we call PWGAN, combining the Deep Convolutional GAN (DCGAN) and Wasserstein GAN with Gradient Penalty (WGAN-GP) together in the structure, and the other, which we call CNN-PGAN, applying a pre-trained CNN as a discriminator to the parallel GAN. Both PWGAN and CNN-PGAN consist of a number of discriminators and generators according to the number of target categories. Experimental results on the TerraSAR-X single polarization dataset demonstrate the effectiveness of the proposed method.

A novel three-step method to reduce speckle in Synthetic Aperture Radar (SAR) images is presented. First, an edge detector is used to detect the edge orientations in SAR images. Second, using a data processing method named empirical mode decomposition, the SAR image is smoothed along four directions: horizontal, vertical, left diagonal and right diagonal, respectively. Third, the SAR image is reconstructed such that if the edge direction is horizontal, the pixel smoothed along the horizontal direction is used to reconstruct the image, and so on; if an edge is not detected, the mean of the four directionally smoothed images is used to reconstruct the image. Application of this method to SAR images has shown that it is well balanced in the quality of visual appearance, mean preservation, edge preservation, and reduction of the standard deviation.

ABSTRACTThe speckle noise of synthetic aperture radar (SAR) images limits its application in change detection. Compared with improved ratio (IR) and log-ratio (LR) operators, the neighborhood-based ratio (NR) technique can restrain the influence of speckle noise and is more suitable for change detection in SAR images. However, we find three drawbacks of NR by analyzing this method carefully. To overcome these defects, we propose an improved neighborhood-based ratio (INR) approach for change detection in SAR images. INR restructures the NR operator to exploit the neighborhood information more reasonably and is expected to reduce the impact of speckle noise better. IR, LR, mean ratio operator, NR, and INR are tested on two data sets to compare their performances in change detection of SAR images. Experimental results show that the proposed method can obtain better difference image than other state-of-art methods and improve the accuracy of change detection in SAR images effectively.

<p>The velocity of glaciers is commonly derived by offset tracking using pairwise cross correlation or feature matching of either optical or synthetic aperture radar (SAR) images.  SAR images, however, are inherently affected by noise-like radar speckle and require therefore much larger images patches for successful tracking compared to the patch size used with optical data. As a consequence, glacier velocity maps based on SAR offset tracking have a relatively low resolution compared to the nominal resolution of SAR sensors. Moreover, tracking may fail because small features on the glacier surface cannot be detected due to radar speckle. Although radar speckle can be reduced by applying spatial low-pass filters (e.g. 5x5 boxcar), the spatial smoothing reduces the image resolution roughly by an order of magnitude which strongly reduces the tracking precision. Furthermore, it blurs out small features on the glacier surface, and therefore tracking can also fail unless clear features like large crevasses are visible.</p><p>In order to create high resolution velocity maps from SAR images and to generate speckle-free radar images of glaciers, we present a new method that derives the glacier surface velocity field by correlating temporally averaged sub-stacks of a series of SAR images. The key feature of the method is to warp every pixel in each SAR image according to its temporally increasing offset with respect to a reference date. The offset is determined by the glacier velocity which is obtained by maximizing the cross-correlation between the averages of two sub-stacks. Currently, we need to assume that the surface velocity is constant during the acquisition period of the image series but this assumption can be relaxed to a certain extend.</p><p>As the method combines the information of multiple images, radar speckle are highly suppressed by temporal multi-looking, therefore the signal-to-noise ratio of the cross-correlation is significantly improved. We found that the method outperforms the pair-wise cross-correlation method for velocity estimation in terms of both the coverage and the resolution of the velocity field. At the same time, very high resolution radar images are obtained and reveal features that are otherwise hidden in radar speckle.</p><p>As the reference date, to which the sub-stacks are averaged, can be arbitrarily chosen a smooth flow animation of the glacier surface can be generated based on a limited number of SAR images. The presented method could build a basis for a new generation of tracking methods as the method is excellently suited to exploit the large number of emerging free and globally available high resolution SAR image time series.</p>

We address the application of model-supported exploitation techniques to synthetic aperture radar (SAR) imagery. The emphasis is on monitoring SAR imagery using wide area 2D and/or 3D site models along with contextual information. We consider here the following tasks useful in monitoring: (a) site model construction using segmentation and labeling techniques, (b) target detection, (c) target classification and indexing, and (d) SAR image-site model registration. The 2-D wide area site models used here for SAR image exploitation differ from typical site models developed for RADIUS applications, in that they do not model specific facilities, but constitute wide area site models of cultural features such as urban clutter areas, roads, clearings, fields, etc. These models may be derived directly from existing site models, possibly constructed from electro-optical (EO) observations. When such models are not available, a set of segmentation and labeling techniques described here can be used for the construction of 2D site models. The use of models can potentially yield critical information which can disambiguate target signatures in SAR images. We address registration of SAR and EO images to a common site model. Specific derivations are given for the case of registration within the RCDE platform. We suggest a constant false alarm rate (CFAR) detection scheme and a topographic primal sketch (TPS) based classification scheme for monitoring target occurrences in SAR images. The TPS of an observed target is matched against candidate targets TPSs synthesized for the preferred target orientation, inferred from context (e.g. road or parking lot targets). Experimental results on real and synthetic SAR images are provided.

In single band and single polarized synthetic aperture radar (SAR) images, the information is limited to intensity and texture only and it is very difficult to interpret such SAR images without any a priori information. For unsupervised classification of SAR images, M-band wavelet decomposition is performed on the SAR image and sub-band selection on the basis of energy levels is applied to improve the classification results since sparse representation of sub-bands degrades the performance of classification. Then, textural features are obtained from selected sub-bands and integrated with intensity features. An adaptive neuro-fuzzy algorithm is used to improve computational efficiency by extracting significant features. K-means classification is performed on the extracted features and land features are labeled. This classification algorithm involves user defined parameters. To remove the user dependency and to obtain maximum achievable classification accuracy, an algorithm is developed in this paper for classification accuracy in terms of the parameters involved in the segmentation process. This is very helpful to develop the automated land-cover monitoring system with SAR, where optimized parameters are to be identified only once and these parameters can be applied to SAR imagery of the same scene obtained year after year. A single band, single polarized SAR image is classified into water, urban and vegetation areas using this method and overall classification accuracy is obtained in the range of 85.92%–93.70% by comparing with ground truth data.