Signal and Image Processing for Remote Sensing

AN ASSESSMENT OF SPATIAL VARIATION OF LAND SURFACE CHARACTERISTICS OF MINNA, NIGER STATE NIGERIA FOR SUSTAINABLE URBANIZATION USING GEOSPATIAL TECHNIQUES

Limited resolution is one of the most important factors hindering the application of remote sensing images (RSIs). Single-image super resolution (SISR) is a technique to improve the spatial resolution of digital images and has attracted the attention of many researchers. In recent years, with the advancement of deep learning (DL) frameworks, many DL-based SISR models have been proposed and achieved state-of-the-art performance; however, most SISR models for RSIs use the bicubic downsampler to construct low-resolution (LR) and high-resolution (HR) training pairs. Considering that the quality of the actual RSIs depends on a variety of factors, such as illumination, atmosphere, imaging sensor responses, and signal processing, training on “ideal” datasets results in a dramatic drop in model performance on real RSIs. To address this issue, we propose to build a more realistic training dataset by modeling the degradation with blur kernels and imaging noises. We also design a novel residual balanced attention network (RBAN) as a generator to estimate super-resolution results from the LR inputs. To encourage RBAN to generate more realistic textures, we apply a UNet-shape discriminator for adversarial training. Both referenced evaluations on synthetic data and non-referenced evaluations on actual images were carried out. Experimental results validate the effectiveness of the proposed framework, and our model exhibits state-of-the-art performance in quantitative evaluation and visual quality. We believe that the proposed framework can facilitate super-resolution techniques from research to practical applications in RSIs processing.

A benchmark data set for aircraft type recognition from remote sensing images

Sufficient remote sensing observation data during crop main growing season is of great importance in improving the accuracy of data assimilation of crop model. The optical remote sensing data are susceptible to cloud and rain, so the amount of clear optical data is very limited in cloudy weather or rainy day. Synthetic Aperture Radar (SAR) is not dependent on cloud cover or light conditions, it can penetrate through clouds and have all-weather capabilities. This allows for a more reliable and consistent crop monitoring and yield estimation in terms of radar sensor data. So, the aim of this article is to improve the accuracy for winter wheat yield estimation by joint assimilation of SAR and optical satellite images into crop model. In this study, SAR images are acquired by C-band SAR sensor boarded on Sentinel-1 satellites, and optical images are obtained from Sentinel-2 satellites. Remote sensing data and ground data are all collected during the main growth and development stages of winter wheat. Both normalized difference vegetation index (NDVI) derived from Sentinel-2 images and backscattering coefficients and polarimetric indicators computed from Sentinel-1 images are used in water cloud model to derive soil moisture (SM) time series images. To improve the prediction of crop yields at filed scale, we incorporate remotely sensed soil moisture into the WOrld FOod STudies (WOFOST) model using Ensemble Kalman filter (EnKF) algorithm. In general, the results show that data assimilation schemes of remotely sensed soil moisture slightly improved the correlation of observed and simulated yields (R2 = 0.30; RMSE =782 kg ha−1) compared to the situation without data assimilation (R2 = 0.14; RMSE = 1398 kg ha−1). Results of this study indicate that the potential for assimilation SAR and optical remote sensing data to improve field yield estimates is relatively low, limitations are due to insufficient no-cloud optical remotely sensed data and root zone soil moisture information. Consequently, the results of this study demonstrate the potential and usefulness of assimilating both SAR and optical remote sensing data into crop model for crop monitoring and yield estimation. Moreover, this also provides the reference for crop yield estimation with data assimilation in other agricultural landscapes.

To overcome the difficulty of automating and intelligently classifying the ground features in remote-sensing hyperspectral images, machine learning methods are gradually introduced into the process of remote-sensing imaging. First, the PaviaU, Botswana, and Cuprite hyperspectral datasets are selected as research subjects in this study, and the objective is to process remote-sensing hyperspectral images via machine learning to realize the automatic and intelligent classification of features. Then, the basic principles of the support vector machine (SVM) and extreme learning machine (ELM) classification algorithms are introduced, and they are applied to the datasets. Next, by adjusting the parameter estimates using a restricted Boltzmann machine (RBM), a new terrain classification model of hyperspectral images that is based on a deep belief network (DBN) is constructed. Next, the SVM, ELM, and DBN classification algorithms for hyperspectral image terrain classification are analysed and compared in terms of accuracy and consistency. The results demonstrate that the average detection accuracies of ELM on the three datasets are 89.54%, 96.14%, and 96.28%, and the Kappa coefficient values are 0.832, 0.963, and 0.924; the average detection accuracies of SVM are 88.90%, 92.11%, and 91.68%, and the Kappa coefficient values are 0.768, 0.913, and 0.944; the average detection accuracies of the DBN classification model are 92.36%, 97.31%, and 98.84%, and the Kappa coefficient values are 0.883, 0.944, and 0.972. The results also demonstrate that the classification accuracy of the DBN algorithm exceeds those of the previous two methods because it fully utilizes the spatial and spectral information of hyperspectral remote-sensing images. In summary, the DBN algorithm that is proposed in this study has high application value in object classification for remote-sensing hyperspectral images.

This article made a detailed introduction to the design and implementation of image processing algorithm framework, with its focus on data-driven, image processing algorithm, UI and processing dialog optimization. Implemented by the C++ programming language, the final framework provides a new way and reference to the Customizable development of remote sensing image processing algorithm.The remote sensing image processing system is a set of technologies and methodoloies that can add enormous value in handling the problem of the earth sciences and related fields.Most practitioners are, primarily, professionals in fields such as hydrology, agriculture, engineering, forestry, geology and urban studies; giving them the writing remote sensing processing algorithm skills considerably enhances their expertise and capablities.

Remote Sensing (RS) image classification has recently attracted great attention for its application in different tasks, including environmental monitoring, battlefield surveillance, and geospatial object detection. The best practices for these tasks often involve transfer learning from pre-trained Convolutional Neural Networks (CNNs). A common approach in the literature is employing CNNs for feature extraction, and subsequently train classifiers exploiting such features. In this paper, we propose the adoption of transfer learning by fine-tuning pre-trained CNNs for end-to-end aerial image classification. Our approach performs feature extraction from the fine-tuned neural networks and remote sensing image classification with a Support Vector Machine (SVM) model with linear and Radial Basis Function (RBF) kernels. To tune the learning rate hyperparameter, we employ a linear decay learning rate scheduler as well as cyclical learning rates. Moreover, in order to mitigate the overfitting problem of pre-trained models, we apply label smoothing regularization. For the fine-tuning and feature extraction process, we adopt the Inception-v3 and Xception inception-based CNNs, as well the residual-based networks ResNet50 and DenseNet121. We present extensive experiments on two real-world remote sensing image datasets: AID and NWPU-RESISC45. The results show that the proposed method exhibits classification accuracy of up to 98%, outperforming other state-of-the-art methods.

Corn is an important food crop worldwide, and its yield is directly related to Chinese food security. Accurate remote sensing extraction of corn can realize the rational application of land resources, which is of great significance to the sustainable development of modern agriculture. In the field of large-scale crop remote sensing classification, single-period optical remote sensing images often cannot achieve high-precision classification. To improve classification accuracy, multiple time series image combinations have gradually been adopted. However, due to the influence of cloudy and rainy weather, it is often difficult to obtain complete time series optical images. Synthetic aperture radar (SAR) data are imaged by microwaves, which have strong penetrating power and are not affected by clouds. A critical way to solve this problem is to use SAR images to compensate for the lack of optical images and obtain a complete time series image in the corn-growing season. However, SAR images have limited wavelengths and cannot provide important wavelengths, such as visible light bands and near-infrared information. To solve this problem, this study took Zhaodong City, a vital corn-planting base in China, as the research area; took GF-6/GF-3 and Sentinel-1/Sentinel-2 as remote sensing data sources; designed12 classification scenarios; analyzed the best classification period and the best time series combination of corn classification; studied the influence of SAR images on the classification results of time series images; and compared the classification differences between GF-6/GF-3 and Sentinel-1/Sentinel-2. The results show that the classification accuracy of time series combinations is much higher than that of single-period images. The polarization characteristics of SAR images can improve the classification accuracy with time series images. The classification accuracy of GF series images from China is obviously higher than that of Sentinel series images. The research performed in this paper can provide a reference for agricultural classification by using remote sensing data.

Synthetic aperture radar (SAR) images provide all-weather and all-time capabilities for Earth observation, which becomes highly beneficial in the field of intelligent remote sensing (RS) image interpretation. Due to these advantages, SAR images have been widely exploited in automatic building segmentation tasks under poor weather conditions, especially when disasters happen. However, compared to optical images, the semantics inherent to SAR images are less rich and interpretable due to factors such as speckle noise and imaging geometry. In this scenario, most state-of-the-art methods are focused on designing advanced network architectures or loss functions for building footprint extraction. However, few works have been oriented toward improving segmentation performance through knowledge transfer from optical images. In this article, we propose a novel method based on the <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DisOptNet</monospace> network, which can distill the useful semantic knowledge from optical images into a network only trained with SAR data. Specifically, we first analyze the multilevel feature discrepancies between multiple stages of the networks pretrained on the two image modalities. We observe that feature discrepancies start to increase as the encoding stage gradually changes from low level to high level. Based on such observation, we reuse the early stage features and construct parallel convolutional neural network (CNN) branches that are responsible for capturing high-level domain-specific knowledge for each image modality. The optical branch is aimed at mimicking feature generation at the optical pretrained network given the input SAR images. Then, an aggregation module is introduced to calibrate and fuse the features from different modalities while generating the building segments. Extensive experiments were conducted on a large-scale multisensor all-weather building segmentation dataset with state-of-the-art methods used for comparison. Our experimental results validate the effectiveness of <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DisOptNet</monospace> , which demonstrates great potential in the task of weather-independent building footprint generation under real scenarios. The codes of this article will be made publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/jiankang1991/TGRS_DisOptNet</uri> .

Synthetic aperture radar (SAR) image classification is an important part in the understanding and interpretation of SAR images. Each patch in SAR images has a scene category, but usually contains multiple land-cover classes or latent properties, which can be represented by topics in the probabilistic topic model (PTM). The representation and selection of discriminative features in PTM have a large impact on the classification results. Most of the existing feature learning methods do not make full use of high-level structure feature and the feature correlation within similar images to mine discriminative features. Therefore, this article proposes a discriminative sketch topic model with structural constraint (C-SSTM) for SAR image classification. In the proposed model, each image patch is characterized by structural and texture features. In particular, the sketch structural feature is based on the sketch map to represent the image local structure pattern. Then, the local image manifold information is preserved in terms of structure and texture. In the structural constraint, the texture and structure of each image patch are combined to learn discriminative latent semantic topics between image patches. Finally, each image patch is quantified by discriminative latent semantic topics instead of low-level representation. The experimental results tested on synthetic and real SAR images demonstrate that the proposed C-SSTM is able to learn effective structural feature representation from SAR images. Compared with other related approaches, C-SSTM produces competitive classification accuracies with high time efficiency.

Convolutional neural networks (CNNs) have been applied to learn spatial features for high-resolution (HR) synthetic aperture radar (SAR) image classification. However, there has been little work on integrating the unique statistical distributions of SAR images which can reveal physical properties of terrain objects, into CNNs in a supervised feature learning framework. To address this problem, a novel end-to-end supervised classification method is proposed for HR SAR images by considering both spatial context and statistical features. First, to extract more effective spatial features from SAR images, a new deep spatial context encoder network (DSCEN) is proposed, which is a lightweight structure and can be effectively trained with a small number of samples. Meanwhile, to enhance the diversity of statistics, the nonstationary joint statistical model (NS-JSM) is adopted to form the global statistical features. Specifically, SAR images are transformed into the Gabor wavelet domain and the produced multi-subbands magnitudes and phases are modeled by the log-normal and uniform distribution. The covariance matrix is further utilized to capture the inter-scale and intra-scale nonstationary correlation between the statistical subbands and make the joint statistical features more compact and distinguishable. Considering complementary advantages, a feature fusion network (Fusion-Net) base on group compression and smooth normalization is constructed to embed the statistical features into the spatial features and optimize the fusion feature representation. As a result, our model can learn the discriminative features and improve the final classification performance. Experiments on four HR SAR images validate the superiority of the proposed method over other related algorithms.

In single band and single polarized synthetic aperture radar (SAR) images, the information is limited to intensity and texture only and it is very difficult to interpret such SAR images without any a priori information. For unsupervised classification of SAR images, M-band wavelet decomposition is performed on the SAR image and sub-band selection on the basis of energy levels is applied to improve the classification results since sparse representation of sub-bands degrades the performance of classification. Then, textural features are obtained from selected sub-bands and integrated with intensity features. An adaptive neuro-fuzzy algorithm is used to improve computational efficiency by extracting significant features. K-means classification is performed on the extracted features and land features are labeled. This classification algorithm involves user defined parameters. To remove the user dependency and to obtain maximum achievable classification accuracy, an algorithm is developed in this paper for classification accuracy in terms of the parameters involved in the segmentation process. This is very helpful to develop the automated land-cover monitoring system with SAR, where optimized parameters are to be identified only once and these parameters can be applied to SAR imagery of the same scene obtained year after year. A single band, single polarized SAR image is classified into water, urban and vegetation areas using this method and overall classification accuracy is obtained in the range of 85.92%–93.70% by comparing with ground truth data.

Modern remote sensing (RS) image application systems often distribute image processing tasks among multiple data centers and then gather the processed images from each center to efficiently synthesize the final product. In this paper, we exploit the edge-cloud architecture to design and implement a novel RS image service system, called RS-pCloud, which leverages the Peer-to-Peer (P2P) model to integrate multiple data centers and their associated edge networks. The data center as cloud platform is responsible for the storage and processing of original RS images, as well as the storage of partial processed images while the edge network is mainly for caching and sharing the processed images. With this design, RS-pCloud not only achieves the load sharing of processing works but also attains the data efficiency among the edges at the same time, which in turn improves the performance of the image processing and reduce the cost of the data transmission as well. RS-pCloud is designed to be used in a transparent way where it receives a query task from the user through a certain cloud platform, split the task into different sub-tasks, according to the location of the data they required, and then distribute the sub-tasks to corresponding clouds for near-data processing, the returned results from each cloud are first cached in specific edge for further sharing and then gathered at the client to synthesize the final product. We implemented and deployed RS-pCloud on three clusters in conjunction with an edge network to show its performance advantages over traditional single-cluster systems.

Thanks to the availability of large-scale data, deep Convolutional Neural Networks (CNNs) have witnessed success in various applications of computer vision. However, the performance of CNNs on Synthetic Aperture Radar (SAR) image classification is unsatisfactory due to the lack of well-labeled SAR data, as well as the differences in imaging mechanisms between SAR images and optical images. Therefore, this paper addresses the problem of SAR image classification by employing the Generative Adversarial Network (GAN) to produce more labeled SAR data. We propose special GANs for generating SAR images to be used in the training process. First, we incorporate the quadratic operation into the GAN, extending the convolution to make the discriminator better represent the SAR data; second, the statistical characteristics of SAR images are integrated into the GAN to make its value function more reasonable; finally, two types of parallel connected GANs are designed, one of which we call PWGAN, combining the Deep Convolutional GAN (DCGAN) and Wasserstein GAN with Gradient Penalty (WGAN-GP) together in the structure, and the other, which we call CNN-PGAN, applying a pre-trained CNN as a discriminator to the parallel GAN. Both PWGAN and CNN-PGAN consist of a number of discriminators and generators according to the number of target categories. Experimental results on the TerraSAR-X single polarization dataset demonstrate the effectiveness of the proposed method.

The dynamics of the Rhine outflow plume in the proximity of the river mouth is investigated by using remote sensing data and numerical simulations. The remote sensing data consist of 41 synthetic aperture radar (SAR) images acquired by the First and Second European Remote Sensing satellites ERS-1 and ERS-2 over the outflow region of the river Rhine. Most of them show sea surface signatures of oceanic phenomena, for example, surface current and wind variations, ship wakes, and oil slicks. In particular, in 36 of these images pronounced frontal features are visible as narrow zones of mainly enhanced, sometimes enhanced/reduced radar backscatter that can be associated with the Rhine surface front. Within the area enclosed by the frontal line, large zones characterized by a lower radar backscatter than in the outer area are often visible. The analysis of the ERS SAR images suggests that the form and the location of the frontal features are mainly linked to the semidiurnal tidal phase in the outflow region, although their variability suggests also that they weakly depend on river discharge, residual currents, and neap-spring tidal cycle. In order to test this observational hypothesis, the results obtained from the analysis of the ERS SAR images are compared with the results obtained from the numerical simulation of the hydrodynamics of the Rhine outflow region carried out using a two-layer, frontal model, which is based on the nonlinear, hydrostatic shallow-water equations on an f plane. The model is forced by prescribing tidal and residual currents and river discharge at the open boundaries. Several simulations are performed by varying the values of these forcing parameters. The numerical results corroborate the observational conjecture: It is found that the form and the location of the simulated interface outcropping lines in the proximity of the river mouth are mainly determined by the semidiurnal tidal phase in the outflow region and that river discharge, residual currents, and neap-spring tidal cycle contribute only secondarily to their determination. Inserting the simulated surface velocity field into a simple radar-imaging model that relates the modulation of the backscattered radar power to the surface velocity convergence in radar look direction, narrow, elongated bands of enhanced radar backscatter emerge near the model frontal line while patches of low radar backscatter appear within the simulated Rhine plume area. The consistency of the model results with the results obtained from the analysis of the SAR images enables one to infer a mean spatial and temporal evolution of the Rhine outflow plume over a semidiurnal tidal cycle from the analysis of spaceborne SAR images acquired during different tidal cycles over the Rhine outflow area and suggests the possibility of using numerical modeling, in conjunction with the analysis of spaceborne measurements, for monitoring the oceanic variability in the Rhine outflow area.