A New GMRF Self-supervised Algorithm Applied to SAR Image Classification

Conventional application of convolutional neural networks (CNNs) for image classification and recognition is based on the assumption that all target classes are equal (i.e. no hierarchy) and exclusive of one another (i.e. no overlap). CNN-based image classifiers built on this assumption, therefore, cannot take into account an innate hierarchy among target classes (e.g. cats and dogs in animal image classification) or additional information that can be easily derived from the data (e.g. numbers larger than five in the recognition of handwritten digits), thereby resulting in scalability issues when the number of target classes is large. Combining two related but slightly different ideas of hierarchical classification and logical learning by auxiliary inputs, we propose a new learning framework called hierarchical auxiliary learning, which not only address the scalability issues with a large number of classes but also could further reduce the classification/recognition errors with a reasonable number of classes. In the hierarchical auxiliary learning, target classes are semantically or non-semantically grouped into superclasses, which turns the original problem of mapping between an image and its target class into a new problem of mapping between a pair of an image and its superclass and the target class. To take the advantage of a superclass as a hint during the learning phase, we introduce an auxiliary block into a neural network, which generates auxiliary scores used as additional information for final classification/recognition; in this paper, we add the auxiliary block between the last residual block and the fully-connected output layer of the ResNet. Experimental results show that the proposed hierarchical auxiliary learning reduces classification errors up to 0.56, 1.6 and 3.56 percent with MNIST, SVHN and CIFAR-10 datasets, respectively.

The state-of-the-art object detection and image classification methods can perform impressively on more than 9k classes. In contrast, the number of classes in semantic segmentation datasets is relatively limited. This is not surprising when the restrictions caused by the lack of labeled data and high computation demand for segmentation are considered. In this paper, we propose a novel training methodology to train and scale the existing semantic segmentation models for a large number of semantic classes without increasing the memory overhead. In our embedding-based scalable segmentation approach, we reduce the space complexity of the segmentation model’s output from O(C) to O(1), propose an approximation method for ground-truth class probability, and use it to compute cross-entropy loss. The proposed approach is general and can be adopted by any state-of-the-art segmentation model to gracefully scale it for any number of semantic classes with only one GPU. Our approach achieves similar, and in some cases, even better mIoU for Cityscapes, Pascal VOC, ADE20k, COCO-Stuff10k datasets when adopted to DeeplabV3+ model with different backbones. We demonstrate a clear benefit of our approach on a dataset with 1284 classes, bootstrapped from LVIS and COCO annotations, with almost three times better mIoU than the DeeplabV3+. Our source code is available at: https://github.com/shipra25jain/ESSNet.

Recently, Convolutional Neural Network (CNN) has achieved some success in synthetic aperture radar (SAR) image classification. This outstanding performance mainly depends on a large number of training samples, but achieved satisfactory classification results with limited training samples remains a challenge. To address this problem, we propose a dual-scale Siamese densely connected network with Markov Random Fields (DS-SDCNet-MRF) for single-polarization SAR image classification. First, the Siamese densely connected network (SDCNet) is proposed to fully extract discriminative features under limited samples. Then, the proposed DS-SDCNet is constructed with two SDCNets of different scales to produce complementary classification results. Among them, the large-scale SDCNet has a better classification result in the homogeneous region, while the small-scale SDCNet tends to provide good detail preservation. Finally, an improved MRF model which combined the category probability information is proposed to further improve the classification performance. Experimental results on simulated and real single-polarization SAR data demonstrate that the proposed method achieves more encouraging classification performance than the current state-of-the-art classification methods with limited samples.

Unsupervised Bayesian image segmentation using orthogonal series

The recent research of hyperspectral images(HSIs) classification depicts that taking spectral-spatial features into account can considerably improve accuracy. Since HSI is a 3D cube datum, amounts of 3D network structures emerged to extract spectral-spatial features consecutively. In this paper, we present a simple spectral-spatial classification framework(SSP3DNet) based on densely connected structure with Pseudo-3D block for HSIs. Firstly, a data augmentation strategy was implemented to solve the problem of limited and uneven training samples in the data preprocessing step. Secondly, the Pseudo-3D block can capture both spectral and spatial features simultaneously which is more economic by decreasing the number of parameters compared with traditional 3D convolution network. Then DenseNet learning framework is utilized to ease the training of networks as well as improving the classification performance. Especially, to prevent over-fitting, some tricks like early stopping, LeakyReLu, batch normalization, and dropout layers are used in our SSP3DNet which enable the SSP3DNet to obtain accuracy within 80 epochs. Experimental results on two well-known hyperspectral datasets show that the proposed SSP3DNet method achieves the best classification accuracy in comparison with lately traditional and deep-learning-based methods.

The lack of training samples remains one of the major obstacles in applying convolutional neural networks (CNNs) to the hyperspectral image (HSI) classification. In this letter, the accurate classification of HSI with limited training samples is investigated. Due to the advantages of minimizing the distance between samples in the same class and maximizing the distance between samples in different classes, siamese CNN is used for HSI classification with limited training samples. After that, to improve the classification performance, data augmentation is investigated for siamese CNN-based HSI classification. Specifically, pair data augmentation based on CutMix is proposed to generate the training pairs of the same or different classes and the new generated training pairs are used to train siamese CNN. Traditional data augmentation methods simply generate new training samples. It is not proper when data augmentation methods keep the loss function and change the content of the input at the same time. Therefore, a soft-loss-based siamese CNN, which changes its loss according to the coupled replacement data augmentation, is proposed to further address the HSI classification with limited training samples. In the experimental part, on two widely used hyperspectral datasets, the influences of different training samples and window sizes are discussed, and the experimental results reveal that the proposed soft-augmentation-based siamese CNN provides competitive results with limited training samples compared with state-of-the-art methods.

Traditional hyperspectral image (HSI) classification methods mainly include machine learning and convolutional neural network. However, they extremely depend on the large training samples. To obtain high accuracy on limited training samples, we propose a novel end-to-end spectral-spatial multi-scale network (SSMSN) for HSI classification. The SSMSN uses the multi-scale spectral module and the multi-scale spatial module to extract discriminative multi-scale spectral and multi-scale spatial features separately. In the multi-scale spectral module and spatial module, the multi-scale Res2Net block structure can learn multi-scale features at a granular level and increase the range of receptive fields by constructing the hierarchical residual-like connection within one single residual block. To alleviate the overfitting problem and further improve the classification accuracy on limited training samples, we adopt a simple but effective hinge cross-entropy loss function to train the SSMSN at the dynamic learning rate. A large number of experimental results demonstrate that on the Indiana Pines, University of Pavia, Kennedy Space Center, and Salinas Scene data sets, the proposed SSMSN achieves higher classification accuracy than state-of-the-art methods on limited training samples. Meanwhile, our SSMSN obtains less training and testing time than the popular AUSSC method.

Synthetic aperture radar (SAR) image classification is an important part in the understanding and interpretation of SAR images. Each patch in SAR images has a scene category, but usually contains multiple land-cover classes or latent properties, which can be represented by topics in the probabilistic topic model (PTM). The representation and selection of discriminative features in PTM have a large impact on the classification results. Most of the existing feature learning methods do not make full use of high-level structure feature and the feature correlation within similar images to mine discriminative features. Therefore, this article proposes a discriminative sketch topic model with structural constraint (C-SSTM) for SAR image classification. In the proposed model, each image patch is characterized by structural and texture features. In particular, the sketch structural feature is based on the sketch map to represent the image local structure pattern. Then, the local image manifold information is preserved in terms of structure and texture. In the structural constraint, the texture and structure of each image patch are combined to learn discriminative latent semantic topics between image patches. Finally, each image patch is quantified by discriminative latent semantic topics instead of low-level representation. The experimental results tested on synthetic and real SAR images demonstrate that the proposed C-SSTM is able to learn effective structural feature representation from SAR images. Compared with other related approaches, C-SSTM produces competitive classification accuracies with high time efficiency.

Convolutional neural networks (CNNs) have been applied to learn spatial features for high-resolution (HR) synthetic aperture radar (SAR) image classification. However, there has been little work on integrating the unique statistical distributions of SAR images which can reveal physical properties of terrain objects, into CNNs in a supervised feature learning framework. To address this problem, a novel end-to-end supervised classification method is proposed for HR SAR images by considering both spatial context and statistical features. First, to extract more effective spatial features from SAR images, a new deep spatial context encoder network (DSCEN) is proposed, which is a lightweight structure and can be effectively trained with a small number of samples. Meanwhile, to enhance the diversity of statistics, the nonstationary joint statistical model (NS-JSM) is adopted to form the global statistical features. Specifically, SAR images are transformed into the Gabor wavelet domain and the produced multi-subbands magnitudes and phases are modeled by the log-normal and uniform distribution. The covariance matrix is further utilized to capture the inter-scale and intra-scale nonstationary correlation between the statistical subbands and make the joint statistical features more compact and distinguishable. Considering complementary advantages, a feature fusion network (Fusion-Net) base on group compression and smooth normalization is constructed to embed the statistical features into the spatial features and optimize the fusion feature representation. As a result, our model can learn the discriminative features and improve the final classification performance. Experiments on four HR SAR images validate the superiority of the proposed method over other related algorithms.

In single band and single polarized synthetic aperture radar (SAR) images, the information is limited to intensity and texture only and it is very difficult to interpret such SAR images without any a priori information. For unsupervised classification of SAR images, M-band wavelet decomposition is performed on the SAR image and sub-band selection on the basis of energy levels is applied to improve the classification results since sparse representation of sub-bands degrades the performance of classification. Then, textural features are obtained from selected sub-bands and integrated with intensity features. An adaptive neuro-fuzzy algorithm is used to improve computational efficiency by extracting significant features. K-means classification is performed on the extracted features and land features are labeled. This classification algorithm involves user defined parameters. To remove the user dependency and to obtain maximum achievable classification accuracy, an algorithm is developed in this paper for classification accuracy in terms of the parameters involved in the segmentation process. This is very helpful to develop the automated land-cover monitoring system with SAR, where optimized parameters are to be identified only once and these parameters can be applied to SAR imagery of the same scene obtained year after year. A single band, single polarized SAR image is classified into water, urban and vegetation areas using this method and overall classification accuracy is obtained in the range of 85.92%–93.70% by comparing with ground truth data.

Contextual feature extraction is studied for polarimetric synthetic aperture radar (PolSAR) image classification in this work. The contextual locality preserving projection (CLPP) method is proposed for generation of contextual feature cubes using limited training samples. The local information in neighborhood regions is used to extend the training set by including the spatial information. Then, a supervised transform is applied to the polarimetric-contextual feature cube to reduce data dimensionality while preserves the local structures and settles the samples belonging to the same class close together. The classification results on two real L-band PolSAR data from AIRSAR show superior performance of CLPP for PolSAR classification in small sample size situations.

Accurate optical and synthetic aperture radar (SAR) image registration is crucial to multisensor remote sensing applications. Though several algorithms have been proposed, the practical implementation that directly matches large and high-resolution optical and SAR images remains underexplored by the community. Equipped with the satellite positioning parameters, optical and SAR images can be roughly registered based on geographic coordinates. However, the relative positioning accuracy is still dozens, even hundreds of pixels due to the inaccuracies of sensor parameters and elevation. Consequently, we propose a robust registration algorithm, which consists of two stages, where the horizontal positioning errors can be reduced in the first stage, and then, we fine-tune the correspondences in the second stage. Specifically, we propose a novel template matching method based on the dilated convolutional feature (DCF) and epipolar-oriented phase correlation. DCF is constructed by a depthwise-separable dilated convolution with multichannel gradients, which are generated by the Sobel operator for optical images and a ratio of exponentially weighted averages (ROEWA) operator for SAR images. Due to the large reception field of dilated convolution, DCF can retain invariance even for large relative positioning errors. The epipolar-oriented rectangle template, which stretches along the epipolar line, is then proposed to capture more overlapping areas compared to square templates. Furthermore, the outlier removal is implemented in the coordinate system of optical images to avoid the effect of range compression in SAR images. Inliers are finally used to refine the rational polynomial coefficients (RPCs) based on the bundle adjustment technique. Experimental results on high-resolution optical and SAR image products of various scenarios demonstrate the effectiveness of the proposed registration framework. The relative positioning errors of the refined RPCs can be reduced from hundreds of pixels to the subpixel level.

Thanks to the availability of large-scale data, deep Convolutional Neural Networks (CNNs) have witnessed success in various applications of computer vision. However, the performance of CNNs on Synthetic Aperture Radar (SAR) image classification is unsatisfactory due to the lack of well-labeled SAR data, as well as the differences in imaging mechanisms between SAR images and optical images. Therefore, this paper addresses the problem of SAR image classification by employing the Generative Adversarial Network (GAN) to produce more labeled SAR data. We propose special GANs for generating SAR images to be used in the training process. First, we incorporate the quadratic operation into the GAN, extending the convolution to make the discriminator better represent the SAR data; second, the statistical characteristics of SAR images are integrated into the GAN to make its value function more reasonable; finally, two types of parallel connected GANs are designed, one of which we call PWGAN, combining the Deep Convolutional GAN (DCGAN) and Wasserstein GAN with Gradient Penalty (WGAN-GP) together in the structure, and the other, which we call CNN-PGAN, applying a pre-trained CNN as a discriminator to the parallel GAN. Both PWGAN and CNN-PGAN consist of a number of discriminators and generators according to the number of target categories. Experimental results on the TerraSAR-X single polarization dataset demonstrate the effectiveness of the proposed method.

Synthetic aperture radar (SAR) and optical imaging are different remote sensing methods. Given a SAR image, is it possible to predict what the observed scene looks like in an optical image? Transfer between SAR data and optical data seems to be impossible. However, this article shows examples that by applying deep learning techniques on high resolution airborne SAR images and GoogleEarth optical images, the SAR images and optical images can transfer with each other. The transferring help us to better understand the relationship between SAR and optical image, and can be potentially used to transfer detection or classification algorithms for optical image straightforwardly to be applied on SAR image.

Very-high resolution (VHR) synthetic aperture radar (SAR) images from the last generation satellites such as TerraSAR-X and TanDEM-X exhibits special characteristics, especially in the urban-areas. Consequently, attention is needed on special considerations while developing algorithms for SAR image processing and its applications for automated interpretation. With automatic interpretation we refer to the information extraction and characterization for image categorization, retrieval, segmentation, automated target recognition etc.. In this article we focus our attention to the problem of SAR image categorization. The interest in SAR image categorization in VHR SAR (on the contrary to the pixel-based classification in low-resolution SAR images) has increased with enhanced resolution providing opportunity to carry out a more detailed analysis of targets and objects. SAR image categorization requires generation of a compact feature descriptor which accurately define the image content. A feature descriptor can be generated using ‘parametric’ or ‘non-parametric’ approaches in ‘image’ or ‘image within a transformation space’. The objective of this article is to review some selected techniques for this purpose in form of a methodological classification. Qualitative assessment of selected algorithms is presented.
\n
\n