PM-Net: A Multi-Level Keypoints Detector and Patch Feature Learning Network for Optical and SAR Image Matching

The KAZE algorithm has been popularly used to match the remote sensing images. However, it has limited performance when its standard form is directly applied to match the synthetic aperture radar (SAR) images containing significant multiplicative speckle noise. In this paper, an effective SAR image matching algorithm is proposed, which is a combination of KAZE, phase congruency, and speckle noise removing anisotropic diffusion. In this algorithm, the ratio-based Phase Congruency (PC) information is used to eliminate the erroneous features, which improves the repeatability rate of the KAZE features. In addition, the scale layers of the input SAR images are constructed by speckle noise removing anisotropic diffusion method which significantly reduces the effect of noise. The proposed method can improve the repeatability of the extracted KAZE features. Moreover, it can give better recall and precision values than the state-of-the-art methods. It gives the root mean square error (RMSE) values in the range of 0.881 to 1.104 pixels in SAR image matching. Experiments on multi-temporal SAR images demonstrate the applicability of the proposed method.

Matching optical and synthetic aperture radar (SAR) images is often challenged by intricate geometric distortion and nonlinear radiation differences, leading to insufficient and unevenly distributed corresponding points. To tackle this issue, we propose a lightweight deep convolutional network with inverted residuals for optical and SAR image matching. Initially, a fully convolutional neural network (FCNN) is designed to extract high-level and semantic features, robustly capturing universal characteristics between optical and SAR images, adept at handling geometric distortion and nonlinear radiation changes. Notably, we integrate a lightweight architecture with inverted residuals into FCNN to adeptly extract local and global contextual information, facilitating feature reuse and minimizing the loss of crucial features. Additionally, a vector-refined module is deployed to refine dense features, filtering out redundant information. Subsequently, a coarse-to-fine strategy is employed to further eliminate gross errors or incorrect matches. Finally, we evaluate the performance of the proposed network in optical and SAR image matching against manually-designed methods and state-of-the-art deep learning techniques. Experimental results demonstrate that our network significantly surpasses existing methods in terms of the number of correct matches and matching accuracy. Specifically, our proposed network achieves at least a 2.8 times increase in correct matches and an 18% improvement in matching accuracy compared to existing methods.

The usage of a hypersonic platform for remote sensing application has promising prospects, especially for hypersonic platform-borne synthetic aperture radar (SAR) imaging. However, the high-speed of hypersonic platform will lead to extreme friction between the platform and air, which will cause the ionization of air. The ionized gas forms the plasma sheath wrapped around the hypersonic platform. The plasma sheath will severely affect the propagation of SAR signal and further affect the SAR imaging. Therefore, hypersonic platform-borne SAR imaging should be studied from a physical perspective. In this paper, hypersonic platform-borne SAR imaging under plasma sheath is analyzed. The SAR signal propagation in plasma sheath is computed using scatter matrix method. The proposed SAR signal model is verified by using a ground experiment system. Moreover, the effect of attenuation caused by plasma sheath on SAR imaging is studied under different SAR parameters and plasma sheath. The result shows that attenuation caused by plasma sheath will degrade the SAR imaging quality and even cause the point and area targets to be submerged into the noise. The real SAR images under plasma sheath also illustrate this phenomenon. Furthermore, by studying imaging results under different SAR and plasma parameters, it can be concluded that the severe degradation of SAR imaging quality appears at condition of high plasma sheath electron density and low SAR carrier frequency. The work in this paper will be beneficial for the study of hypersonic platform-borne SAR imaging and design of hypersonic SAR imaging systems in the future.

Synthetic aperture radar (SAR) image classification is an important part in the understanding and interpretation of SAR images. Each patch in SAR images has a scene category, but usually contains multiple land-cover classes or latent properties, which can be represented by topics in the probabilistic topic model (PTM). The representation and selection of discriminative features in PTM have a large impact on the classification results. Most of the existing feature learning methods do not make full use of high-level structure feature and the feature correlation within similar images to mine discriminative features. Therefore, this article proposes a discriminative sketch topic model with structural constraint (C-SSTM) for SAR image classification. In the proposed model, each image patch is characterized by structural and texture features. In particular, the sketch structural feature is based on the sketch map to represent the image local structure pattern. Then, the local image manifold information is preserved in terms of structure and texture. In the structural constraint, the texture and structure of each image patch are combined to learn discriminative latent semantic topics between image patches. Finally, each image patch is quantified by discriminative latent semantic topics instead of low-level representation. The experimental results tested on synthetic and real SAR images demonstrate that the proposed C-SSTM is able to learn effective structural feature representation from SAR images. Compared with other related approaches, C-SSTM produces competitive classification accuracies with high time efficiency.

In high-resolution (HR) and very-high resolution (VHR) synthetic aperture radar (SAR) images, focus is now on the patch-oriented image categorization in contrast to the pixel-based classification in low-resolution SAR images. SAR image categorization requires the generation of a compact feature descriptor that accurately defines the content of the image patch under consideration. In this paper we propose a parametric feature descriptor generated on the complex-valued SAR image within a transformation space. The fractional Fourier transform (FrFT), has been considered to transform the image pixels of the single-look complex (SLC) SAR images in order to obtain a simpler statistical response. The real and imaginary components of the complex-valued FrFT coefficients have been modelled with generalized Gaussian distribution (GGD). The proposed feature descriptor is compared with a Wavelet-decomposition-based parametric feature descriptor; and with the FrFT-based and Gabor-filter-bank-based non-parametric feature descriptors. Categorization accuracy enhancement is demonstrated over several categories comprising of natural topologies. The experimental database consists of 2000 image patches (of size 200 × 200 pixels) extracted from SLC HR TerraSAR-X scenes.

Monitoring aircraft using synthetic aperture radar (SAR) images is a very important task. Given its coherent imaging characteristics, there is a large amount of speckle interference in the image. This phenomenon leads to the scattering information of aircraft targets being masked in SAR images, which is easily confused with background scattering points. Therefore, automatic detection of aircraft targets in SAR images remains a challenging task. For this task, this paper proposes a framework for speckle reduction preprocessing of SAR images, followed by the use of an improved deep learning method to detect aircraft in SAR images. Firstly, to improve the problem of introducing artifacts or excessive smoothing in speckle reduction using total variation (TV) methods, this paper proposes a new nonconvex total variation (NTV) method. This method aims to ensure the effectiveness of speckle reduction while preserving the original scattering information as much as possible. Next, we present a framework for aircraft detection based on You Only Look Once v8 (YOLOv8) for SAR images. Therefore, the complete framework is called SAR-NTV-YOLOv8. Meanwhile, a high-resolution small target feature head is proposed to mitigate the impact of scale changes and loss of depth feature details on detection accuracy. Then, an efficient multi-scale attention module was proposed, aimed at effectively establishing short-term and long-term dependencies between feature grouping and multi-scale structures. In addition, the progressive feature pyramid network was chosen to avoid information loss or degradation in multi-level transmission during the bottom-up feature extraction process in Backbone. Sufficient comparative experiments, speckle reduction experiments, and ablation experiments are conducted on the SAR-Aircraft-1.0 and SADD datasets. The results have demonstrated the effectiveness of SAR-NTV-YOLOv8, which has the most advanced performance compared to other mainstream algorithms.

In general, synthetic aperture radar (SAR) imaging and image processing are two sequential steps in SAR image processing. Due to the large size of SAR images, most image processing algorithms require image segmentation before processing. However, the existence of speckle noise in SAR images, as well as poor contrast and the uneven distribution of gray values in the same target, make SAR images difficult to segment. In order to facilitate the subsequent processing of SAR images, this paper proposes a new method that combines the back-projection algorithm (BPA) and a first-order gradient operator to enhance the edges of SAR images to overcome image segmentation problems. For complex-valued signals, the gradient operator was applied directly to the imaging process. The experimental results of simulated images and real images validate our proposed method. For the simulated scene, the supervised image segmentation evaluation indexes of our method have more than 1.18%, 11.2% and 11.72% improvement on probabilistic Rand index (PRI), variability index (VI), and global consistency error (GCE). The proposed imaging method will make SAR image segmentation and related applications easier.

Ship detection from synthetic aperture radar (SAR) images is a hot topic, but the difficulty in collecting labeled SAR images may hinder the development of deep-learning-based detection methods. Inspired by the idea of domain adaptation, in this article, we propose a hierarchical similarity alignment neural network (HSANet) for ship detection in SAR images, which is a domain adaptive (DA) approach with optical remote sensing images as training samples. The kernel target of HSANet is to mine and align both the global structure and the local instance information between SAR and optical images, where two modules, structural alignment module (SAM) and prototype alignment module (PAM), are designed to, respectively, conduct two hierarchies of alignment process. In general, SAM attempts to extract the global structure similarity which exists in image-level feature representation, while PAM tends to extract the local shape similarity which is instance-level representation. To be specific, SAM is developed by Fourier-based feature alignment, which tries to describe the similar structural relationship between optical and SAR images. Meanwhile, PAM is proposed based on the conjoint confidence analysis where the instance-level ship representations of the source and target domains is aligned. SAM and PAM work together to construct a hierarchical domain adaptation network for SAR ship detection. Experiments on several public datasets may indicate the effectiveness of the proposed method.

This paper proposes a novel method on Synthetic Aperture Radar (SAR) Image denoising, which is based on context modeling combined with multiscale orthogonal bandlet coefficients. For the main influence of SAR image is multiplicative speckle noise, the logarithm uniform transform is used to convert it to additive noise. In this paper, the multiscale orthogonal bandlet transform is applied to noisy SAR Image. The contextual model is used to compute contextual values of bandlet coefficients on each scale. Therefore, bandlet coefficients are reconstructed into some parts by contextual values. The spatially adaptive soft-threshold (T) is to denoise SAR image in each part. Experimental results demonstrate that this method outperforms the methods of Ehance-Lee (E-lee), contextual model based on wavelet domain (Wavelet-context), and Hidden Markov Tree model bases on wavelet domain (Wavelet-hmt) on there indexes: Floating index (FI), Standard deviation and Edge saving index of vertical direction, by using a real SAR image. However, the method can not only reduce the speckle noise but also preserve edges and high frequency information, and gets considerable visual effect on SAR Image denoising.

Azimuth resolution improvement for spaceborne SAR images with quasi-non-overlapped Doppler bandwidth

Density-based ship detection in SAR images: Extension to a self-similarity perspective

A novel three-step method to reduce speckle in Synthetic Aperture Radar (SAR) images is presented. First, an edge detector is used to detect the edge orientations in SAR images. Second, using a data processing method named empirical mode decomposition, the SAR image is smoothed along four directions: horizontal, vertical, left diagonal and right diagonal, respectively. Third, the SAR image is reconstructed such that if the edge direction is horizontal, the pixel smoothed along the horizontal direction is used to reconstruct the image, and so on; if an edge is not detected, the mean of the four directionally smoothed images is used to reconstruct the image. Application of this method to SAR images has shown that it is well balanced in the quality of visual appearance, mean preservation, edge preservation, and reduction of the standard deviation.

ABSTRACTThe speckle noise of synthetic aperture radar (SAR) images limits its application in change detection. Compared with improved ratio (IR) and log-ratio (LR) operators, the neighborhood-based ratio (NR) technique can restrain the influence of speckle noise and is more suitable for change detection in SAR images. However, we find three drawbacks of NR by analyzing this method carefully. To overcome these defects, we propose an improved neighborhood-based ratio (INR) approach for change detection in SAR images. INR restructures the NR operator to exploit the neighborhood information more reasonably and is expected to reduce the impact of speckle noise better. IR, LR, mean ratio operator, NR, and INR are tested on two data sets to compare their performances in change detection of SAR images. Experimental results show that the proposed method can obtain better difference image than other state-of-art methods and improve the accuracy of change detection in SAR images effectively.

<p>The velocity of glaciers is commonly derived by offset tracking using pairwise cross correlation or feature matching of either optical or synthetic aperture radar (SAR) images.  SAR images, however, are inherently affected by noise-like radar speckle and require therefore much larger images patches for successful tracking compared to the patch size used with optical data. As a consequence, glacier velocity maps based on SAR offset tracking have a relatively low resolution compared to the nominal resolution of SAR sensors. Moreover, tracking may fail because small features on the glacier surface cannot be detected due to radar speckle. Although radar speckle can be reduced by applying spatial low-pass filters (e.g. 5x5 boxcar), the spatial smoothing reduces the image resolution roughly by an order of magnitude which strongly reduces the tracking precision. Furthermore, it blurs out small features on the glacier surface, and therefore tracking can also fail unless clear features like large crevasses are visible.</p><p>In order to create high resolution velocity maps from SAR images and to generate speckle-free radar images of glaciers, we present a new method that derives the glacier surface velocity field by correlating temporally averaged sub-stacks of a series of SAR images. The key feature of the method is to warp every pixel in each SAR image according to its temporally increasing offset with respect to a reference date. The offset is determined by the glacier velocity which is obtained by maximizing the cross-correlation between the averages of two sub-stacks. Currently, we need to assume that the surface velocity is constant during the acquisition period of the image series but this assumption can be relaxed to a certain extend.</p><p>As the method combines the information of multiple images, radar speckle are highly suppressed by temporal multi-looking, therefore the signal-to-noise ratio of the cross-correlation is significantly improved. We found that the method outperforms the pair-wise cross-correlation method for velocity estimation in terms of both the coverage and the resolution of the velocity field. At the same time, very high resolution radar images are obtained and reveal features that are otherwise hidden in radar speckle.</p><p>As the reference date, to which the sub-stacks are averaged, can be arbitrarily chosen a smooth flow animation of the glacier surface can be generated based on a limited number of SAR images. The presented method could build a basis for a new generation of tracking methods as the method is excellently suited to exploit the large number of emerging free and globally available high resolution SAR image time series.</p>

It is expensive and time-consuming to obtain a large number of labeled synthetic aperture radar (SAR) images. In the task of small training data size, the results of target detection on SAR images using deep network approaches are usually not ideal. In this study, considering that optical remote sensing images are much easier to be labeled than SAR images, we assume to have a large number of labeled optical remote sensing images and a small number of labeled SAR images with the similar scenes, propose to transfer knowledge from optical remote sensing images to SAR images, and develop a domain adaptive Faster R-CNN for SAR target detection with small training data size. In the proposed method, in order to make full use of the label information and realize more accurate domain adaptation knowledge transfer, an instance level domain adaptation constraint is used rather than feature level domain adaptation constraint. Specifically, generative adversarial network (GAN) constraint is applied as the domain adaptation constraint in the adaptation module after the proposals of Faster R-CNN to achieve instance level domain adaptation and learn the transferable features. The experimental results on the measured SAR image dataset show that the proposed method has higher detection accuracy in the task of SAR target detection with small training data size than the traditional Faster R-CNN.