SAR-Optical Image Matching by Integrating Siamese U-Net With FFT Correlation

In general, synthetic aperture radar (SAR) imaging and image processing are two sequential steps in SAR image processing. Due to the large size of SAR images, most image processing algorithms require image segmentation before processing. However, the existence of speckle noise in SAR images, as well as poor contrast and the uneven distribution of gray values in the same target, make SAR images difficult to segment. In order to facilitate the subsequent processing of SAR images, this paper proposes a new method that combines the back-projection algorithm (BPA) and a first-order gradient operator to enhance the edges of SAR images to overcome image segmentation problems. For complex-valued signals, the gradient operator was applied directly to the imaging process. The experimental results of simulated images and real images validate our proposed method. For the simulated scene, the supervised image segmentation evaluation indexes of our method have more than 1.18%, 11.2% and 11.72% improvement on probabilistic Rand index (PRI), variability index (VI), and global consistency error (GCE). The proposed imaging method will make SAR image segmentation and related applications easier.

Aiming at the application background of sliding spotlight mode Synthetic Aperture Radar (SAR) imaging processing, this paper builds a System-on-a-Programmable-Chip-based (SoPC-based) sliding spotlight mode SAR imaging verification system, which uses software and hardware collaborative design method to complete algorithm-to-hardware mapping and can quickly and truly perform in FPGA or ASIC. Finally, we analyzed the implementation of the imaging prototype system and the imaging results. For the generation of correlation factors, this paper hierarchically decomposes the factor generation operation to realize the time division multiplexing of the operation unit, in order to reduce resource consumption. At the same time, a variable point Fast Fourier Transformation (FFT) processor based on a variable radix-2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> multiplexing butterfly unit is designed and implemented. Combining these two parts with other modules enables the entire sliding spotlight mode imaging process. On the basis of keeping the error of the result small, the overall resource consumption is greatly reduced. The 1024~32768 point FFT is implemented by the top-level configurable method, and the operation delay is reduced by more than 20% compared with the Xilinx FFT IP core. The final imaging time is reduced to seconds, and the imaging results meet the requirements. Subsequent compatibility design for other SAR imaging modes also can be achieved.

Synthetic aperture radar (SAR) image segmentation is the key to SAR image automatic interpretation. However, speckle noise, intensity inhomogeneity, and irregular shaped objects with changing edge often make the SAR image segmentation very difficult, and existing algorithms have high computational complexity. We propose a region-based level set method using the local image intensity information. To represent the statistical characteristics of speckle noise, we first use a gamma statistical distribution to model every segmented SAR image. We then apply a modified region mean estimation formula to efficiently segment SAR images with inhomogeneity. Finally, Gaussian filtering is employed to regularize the level set function, which can avoid reinitialization. The experimental results on synthetic and real-world SAR images demonstrate that the proposed method has less computation cost, faster convergence rate, and more accurate segmentation results.

Due to the differences in radiation and geometric characteristics of optical and synthetic aperture radar (SAR) images, there is still a huge challenge for accurate matching. In this paper, we propose a patch-matching network (PM-Net) to improve the matching performance of optical and SAR images. First, a multi-level keypoints detector (MKD) with fused high-level and low-level features is presented to extract more robust keypoints from optical and SAR images. Second, we use a two-channel network structure to improve the image patch matching performance. Benefiting from this design, the proposed method can directly learn the similarity between optical and SAR image patches without manually designing features and descriptors. Finally, the MKD and two-channel net-work are trained separately on GL3D and QXS-SAROPT data sets, and the PM-Net is tested on multiple pairs of optical and SAR images. The experimental results demonstrate that the proposed method outperforms four advanced image matching networks on qualitative and quantitative assessments. The quantitative experiment results show that using our method correct matching points numbers are increased by more than 1.15 times, the value of F1-measure is raised by an average of 7.4% and the root mean squared error (RMSE) is reduced by more than 15.3%. The advantages of MKD and the two-channel network are also verified through ablation experiments.

Synthetic Aperture Radar (SAR) image segmentation stands as a formidable research frontier within the domain of SAR image interpretation. The fully convolutional network (FCN) methods have recently brought remarkable improvements in SAR image segmentation. Nevertheless, these methods do not utilize the peculiarities of SAR images, leading to suboptimal segmentation accuracy. To address this issue, we rethink SAR image segmentation in terms of sequential information of transformers and cross-modal features. We first discuss the peculiarities of SAR images and extract the mean and texture features utilized as auxiliary features. The extraction of auxiliary features helps unearth the distinctive information in the SAR images. Afterward, a feature-enhanced FCN with the transformer encoder structure, termed FE-FCN, which can be extracted to context-level and pixel-level features. In FE-FCN, the features of a single-mode encoder are aligned and inserted into the model to explore the potential correspondence between modes. We also employ long skip connections to share each modality’s distinguishing and particular features. Finally, we present the connection-enhanced conditional random field (CE-CRF) to capture the connection information of the image pixels. Since the CE-CRF utilizes the auxiliary features to enhance the reliability of the connection information, the segmentation results of FE-FCN are further optimized. Comparative experiments conducted on the Fangchenggang (FCG), Pucheng (PC), and Gaofen (GF) SAR datasets. Our method demonstrates superior segmentation accuracy compared to other conventional image segmentation methods, as confirmed by the experimental results.

Common feature representation in optical and synthetic aperture radar (SAR) image registration is one of the most challenging tasks due to the significant geometric and radiometric differences. This letter proposed a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Prominent structure-guided feature (PSGF)</i> representation for SAR and optical image registration. Firstly, the prominent structure of the image is highlighted based on windowed inherent variations, which is conducive to identifying more accurate and reliable corresponding points. Secondly, the maximum response index filter banks are proposed to extract structure features with multi-orientation convolution results. Then the structure feature-guided representation generated from this filtering map is quantized in histograms. Finally, the descriptor with radiation invariance is employed for feature matching, enabling automatic image registration with high accuracy. Comparative analysis with state-of-the-art methods on diverse terrain data demonstrates the superiority of the proposed PSGF method for SAR and optical image registration.

The usage of a hypersonic platform for remote sensing application has promising prospects, especially for hypersonic platform-borne synthetic aperture radar (SAR) imaging. However, the high-speed of hypersonic platform will lead to extreme friction between the platform and air, which will cause the ionization of air. The ionized gas forms the plasma sheath wrapped around the hypersonic platform. The plasma sheath will severely affect the propagation of SAR signal and further affect the SAR imaging. Therefore, hypersonic platform-borne SAR imaging should be studied from a physical perspective. In this paper, hypersonic platform-borne SAR imaging under plasma sheath is analyzed. The SAR signal propagation in plasma sheath is computed using scatter matrix method. The proposed SAR signal model is verified by using a ground experiment system. Moreover, the effect of attenuation caused by plasma sheath on SAR imaging is studied under different SAR parameters and plasma sheath. The result shows that attenuation caused by plasma sheath will degrade the SAR imaging quality and even cause the point and area targets to be submerged into the noise. The real SAR images under plasma sheath also illustrate this phenomenon. Furthermore, by studying imaging results under different SAR and plasma parameters, it can be concluded that the severe degradation of SAR imaging quality appears at condition of high plasma sheath electron density and low SAR carrier frequency. The work in this paper will be beneficial for the study of hypersonic platform-borne SAR imaging and design of hypersonic SAR imaging systems in the future.

In recent years, the rapid development of Deep Learning (DL) has provided a new method for ship detection in Synthetic Aperture Radar (SAR) images. However, there are still four challenges in this task. (1) The ship targets in SAR images are very sparse. A large number of unnecessary anchor boxes may be generated on the feature map when using traditional anchor-based detection models, which could greatly increase the amount of computation and make it difficult to achieve real-time rapid detection. (2) The size of the ship targets in SAR images is relatively small. Most of the detection methods have poor performance on small ships in large scenes. (3) The terrestrial background in SAR images is very complicated. Ship targets are susceptible to interference from complex backgrounds, and there are serious false detections and missed detections. (4) The ship targets in SAR images are characterized by a large aspect ratio, arbitrary direction and dense arrangement. Traditional horizontal box detection can cause non-target areas to interfere with the extraction of ship features, and it is difficult to accurately express the length, width and axial information of ship targets. To solve these problems, we propose an effective lightweight anchor-free detector called R-Centernet+ in the paper. Its features are as follows: the Convolutional Block Attention Module (CBAM) is introduced to the backbone network to improve the focusing ability on small ships; the Foreground Enhance Module (FEM) is used to introduce foreground information to reduce the interference of the complex background; the detection head that can output the ship angle map is designed to realize the rotation detection of ship targets. To verify the validity of the proposed model in this paper, experiments are performed on two public SAR image datasets, i.e., SAR Ship Detection Dataset (SSDD) and AIR-SARShip. The results show that the proposed R-Centernet+ detector can detect both inshore and offshore ships with higher accuracy than traditional models with an average precision of 95.11% on SSDD and 84.89% on AIR-SARShip, and the detection speed is quite fast with 33 frames per second.

Synthetic aperture radar (SAR) image segmentation usually involves two crucial issues: suitable speckle noise removing technique and effective image segmentation methodology. Here, an efficient SAR image segmentation method considering both of the two aspects is presented. As for the first issue, the famous nonlocal mean (NLM) filter is introduced in this study to suppress the multiplicative speckle noise in SAR image. Furthermore, to achieve a higher denoising accuracy, the local neighboring pixels in the searching window are projected into a lower dimensional subspace by principal component analysis (PCA). Thus, the nonlocal mean filter is implemented in the subspace. Afterwards, a multi-objective clustering algorithm is proposed using the principals of artificial immune system (AIS) and kernel-induced distance measures. The multi-objective clustering has been shown to discover the data distribution with different characteristics and the kernel methods can improve its robustness to noise and outliers. Experiments demonstrate that the proposed method is able to partition the SAR image robustly and accurately than the conventional approaches.

Reliable and accurate land use/land cover (LULC) map is a crucial data source for the understanding of coupled human–environment systems, monitoring changes, timely low-cost planning, and management of natural resources. Improvements in sensor technologies and machine learning capabilities have shifted the attention of remote sensing community to data complementarity through fusion of multi-sensor data for accurate feature extraction and mapping. Amalgamation of optical and synthetic aperture radar (SAR) images has shown promising advantages in enhancing the accuracy of extracting LULC as such method allows exploitation of information in sensors. This study investigated the potential of using freely available multisource Sentinel images to extract LULC maps in semi-arid environments through multi-scale geographic object-based image analysis (GEOBIA). A multi-scale classification framework that integrates GEOBIA, correlation-based feature selection (CFS), and random forest (RF)-supervised classification was adopted to extract LULC from assimilation of Sentinel multi-sensor products. First, Sentinel-1 and -2 images were pre-processed. Second, optimum multi-scale segmentation levels were selected using F-score segmentation quality measures. Third, 70 features of various spectral indices and derivatives and geometrical features from optical data and multiple ratios and textural features from dual-polarization SAR images were computed, and a CFS based on wrapper approach was used to select the most significant features at multi-scale levels. Finally, a single and multi-scale RF classifier was used to extract LULC classes using the most relevant features extracted from Sentinel SAR and optical images. Results of multi-scale image segmentation optimization showed that scale parameter (SP) values of 40, 60, and 150 were optimal for extraction of LULC classes. Results of feature selection showed that 22, 24, and 27 features were selected at scale SP values of 40, 60, and 150, respectively. Half of the features were common among the three scales. Single RF classification yielded overall accuracy (OA) values of 92.10%, 93%, and 91% and kappa coefficients of 0.901, 0.912, and 0.89 at scale values of 150, 60, and 40, respectively. Multiscale RF classification from scale values of 150 and 60 produced better LULC classification with OA 96.06% and kappa coefficient of 0.95 compared with other scale SP values. The integrated approach demonstrated an effective and promising method for high-quality LULC extraction from coupling optical and SAR images. Overall, multi-sensor Sentinel images along with the adopted approach feature a remarkable potential for improving LULC extraction and can effectively be used to update geographic information system layers for various applications.

This paper proposes a novel method on Synthetic Aperture Radar (SAR) Image denoising, which is based on context modeling combined with multiscale orthogonal bandlet coefficients. For the main influence of SAR image is multiplicative speckle noise, the logarithm uniform transform is used to convert it to additive noise. In this paper, the multiscale orthogonal bandlet transform is applied to noisy SAR Image. The contextual model is used to compute contextual values of bandlet coefficients on each scale. Therefore, bandlet coefficients are reconstructed into some parts by contextual values. The spatially adaptive soft-threshold (T) is to denoise SAR image in each part. Experimental results demonstrate that this method outperforms the methods of Ehance-Lee (E-lee), contextual model based on wavelet domain (Wavelet-context), and Hidden Markov Tree model bases on wavelet domain (Wavelet-hmt) on there indexes: Floating index (FI), Standard deviation and Edge saving index of vertical direction, by using a real SAR image. However, the method can not only reduce the speckle noise but also preserve edges and high frequency information, and gets considerable visual effect on SAR Image denoising.

As synthetic aperture radar (SAR) data is more and more popular in recent days, the fusion of SAR and optical image become inevitable and desirable problem. Image Registration is the fundamental for data fusion, and has been wildly studied. However, literature on registration between SAR and optical image remain little, which means the problem is still challenge and open. On one hand, the SAR and optical image are from totally different sensor with variant imaging mechanism. On the other hand, Simple features could not enough to maintain the accuracy and efficiency of multi-modal registration. We propose a new method for SAR and optical image registration. Different from current methods mainly based on intensity feature or point feature, our method matches two images by their boundary features. A globalized boundary detection algorithm are used for feature extraction. The extracted boundaries were then matched by coherence point drift algorithm. Experiments prove that our method could achieve pixel-level precision, and perform robust for different resolutions.

Traditional deep learning Synthetic Aperture Radar (SAR) image object detection methods fail to provide effective detection results when faced with SAR image datasets with different joint probability distributions obtained from multiple imaging satellites. In this article, an automatic SAR image object detection method based on domain adaptation is proposed to adapt to unlabeled target domain datasets acquired by different satellites. On the basis of introducing an adversarial domain adaptation learning strategy, we propose Adversarial Learning Attention (ALA) and Compensation Loss Module (CLM) on the baseline network. In ALA, considering the great difference in the scattering intensity of SAR images, the entropy vector can be used to distinguish the high-entropy and low-entropy regions among them and assign the corresponding weights, based on which the adversarial domain adaptation learning attention is proposed to achieve instance-level feature alignment and pixel-level feature alignment in source domain and target domain, respectively. In CLM, the domain alignment of pixel-level feature and instance-level feature of SAR image objects is first implemented, and then to make the feature alignment of both domains more accurate, better aggregation of proposals of different classes of prototype objects in the same domain is required, and further compensation loss is proposed to further restrict the prototype alignment of SAR object features in both domains. We conduct experiments on two SAR datasets obtained from different satellites whose results show the superiority of the proposed method over the state-of-the-art (SOTA) methods and the effectiveness of the proposed module in improving the detection accuracy.

Azimuth resolution improvement for spaceborne SAR images with quasi-non-overlapped Doppler bandwidth

We address the application of model-supported exploitation techniques to synthetic aperture radar (SAR) imagery. The emphasis is on monitoring SAR imagery using wide area 2D and/or 3D site models along with contextual information. We consider here the following tasks useful in monitoring: (a) site model construction using segmentation and labeling techniques, (b) target detection, (c) target classification and indexing, and (d) SAR image-site model registration. The 2-D wide area site models used here for SAR image exploitation differ from typical site models developed for RADIUS applications, in that they do not model specific facilities, but constitute wide area site models of cultural features such as urban clutter areas, roads, clearings, fields, etc. These models may be derived directly from existing site models, possibly constructed from electro-optical (EO) observations. When such models are not available, a set of segmentation and labeling techniques described here can be used for the construction of 2D site models. The use of models can potentially yield critical information which can disambiguate target signatures in SAR images. We address registration of SAR and EO images to a common site model. Specific derivations are given for the case of registration within the RCDE platform. We suggest a constant false alarm rate (CFAR) detection scheme and a topographic primal sketch (TPS) based classification scheme for monitoring target occurrences in SAR images. The TPS of an observed target is matched against candidate targets TPSs synthesized for the preferred target orientation, inferred from context (e.g. road or parking lot targets). Experimental results on real and synthetic SAR images are provided.