Cross-Domain Scene Classification Based on a Spatial Generalized Neural Architecture Search for High Spatial Resolution Remote Sensing Images

Markov random field (MRF) based methods have been widely used in high spatial resolution (HSR) image classification. However, many existing MRF-based methods put more emphasis on pixel level contexts while less on superpixel level contextual information. To cope with this issue, this article presents a novel bilevel contextual MRF framework, named BLC-MRF, for HSR imagery classification. Specifically, pixel and superpixel level dependence are incorporated into the proposed MRF model to fully exploit spectral–spatial contextual information and preserve object boundaries in HSR images. In BLC-MRF, a pixel level MRF model is first performed and then cascaded as an input of a superpixel level MRF. In superpixel level, unary and pairwise potential terms are constructed by using the superpixel probability estimation method and spectral histogram distance, respectively. At last, a contextual MRF model is conducted and the final classification map can be computed by using $\alpha$ -expansion algorithm. The benefits of BLC-MRF are twofold: first, the pixel and superpixel level contextual information can be exploited under MRF framework to preserve object boundaries for improving the classification performance, and, second, the algorithm can provide promising results with a small number of training samples. Experimental results on three HSR datasets demonstrate that the proposed approach outperforms several state-of-the-art methods in terms of the classification performance.

A novel fuzzy decision tree is proposed in this paper (the FDT-support vector machine (SVM) classifier), where the node discriminations are implemented via binary SVMs. The tree structure is determined via a class grouping algorithm, which forms the groups of classes to be separated at each internal node, based on the degree of fuzzy confusion between the classes. In addition, effective feature selection is incorporated within the tree building process, selecting suitable feature subsets required for the node discriminations individually. FDT-SVM exhibits a number of attractive merits such as enhanced classification accuracy, interpretable hierarchy, and low model complexity. Furthermore, it provides hierarchical image segmentation and has reasonably low computational and data storage demands. Our approach is tested on two different tasks: natural forest classification using a QuickBird multispectral image and urban classification using hyperspectral data. Exhaustive experimental investigation demonstrates that FDT-SVM is favorably compared with six existing methods, including traditional multiclass SVMs and SVM-based binary hierarchical trees. Comparative analysis is carried out in terms of testing rates, architecture complexity, and computational times required for the operative phase.

Abstract. For high spatial resolution Remote Sensing images, it is very important to investigate the transformational methods between background and target characteristics. Only in this way rich details in images under different imaging conditions can be well extracted. Amongst the characteristics of imagery targets, texture is a visual feature that reflects the homogeneity of images and the inner attributes of different objects. What's more, it includes important information which describes the structural arrangement of objects and the connection with the surrounding environment. This paper regards texture as the major feature and investigates the transformational methods of texture feature description under different imaging conditions. This paper mainly consists of three parts:(1) Construct a wavelet filter based on Gabor wavelet, which describes texture features obtained under different imaging conditions;(2) Process and analyze the different object's texture features jointly by the relationship which is built by the wavelet description;(3) Build the transformation between the wavelet descriptions of the different object's texture features based on the characteristics of the band and direction.

In high spatial resolution (HSR) imagery scene classification, it is a challenging task to recognize the high-level semantics from a large volume of complex HSR images. The probabilistic topic model (PTM), which focuses on modeling topics, has been proposed to bridge the so-called semantic gap. Conventional PTMs usually model the images with a dense semantic representation and, in general, one topic space is generated for all the different features. However, this approach fails to consider the sparsity of the semantic representation, the classification quality, as well as the time consumption. In this paper, to solve the above problems, a fully sparse semantic topic model (FSSTM) framework is proposed for HSR imagery scene classification. FSSTM, with an elaborately designed modeling procedure, is able to represent the image with sparse but representative semantics. Based on this framework, the topic weights of multiple features are exploited by solving a concave maximization problem, which improves the fusion of the discriminative semantic information at the topic level. Meanwhile, the sparsity and representativeness of the topics generated by FSSTM guarantee that the image is adaptive to the change of a topic number. FSSTM can consistently achieve a good performance with a limited number of training samples, and is robust for HSR image scene classification. The experimental results obtained with three different types of HSR image data sets confirm that the proposed algorithm is effective in improving the performance of scene classification, and is highly efficient in discovering the semantics of HSR images when compared with the state-of-the-art PTM methods.

Among the global Earth system changes, human-induced land-cover and land-use changes (LULCC) play significant roles in sustainability of ecosystems. Suburban sprawl, for example, is one of the major components of LULCC that develops urban impervious surface area (ISA) across the landscapes. Increasing ISA is a major environmental issue that has profound impacts on hydrology and climate. Coastal state of Rhode Island has experienced problems associated with urban runoffs due to the alteration of ISA on the landscape. Precise and accurate coverage and location of ISA are key parameters in ISA related environmental studies. Due to the limitations of existing coarse spatial resolution ISA data and current methods for extraction of ISA from high spatial resolution remote sensing imageries, this study developed an algorithm of multiple agent segmentation and classification (MASC) to extract ISA information from high spatial resolution remote sensing imageries. The MASC model includes submodels of segmentation, shadow-effect, MANOV A-based classification, and post-classification. Based on the MASC model, this study built the 1-meter high spatial resolution ISA dataset for the state of Rhode Island, and revealed its spatial patterns. The result indicates that, as of 2004, 10% of the state land has been covered by ISA. The major population centers and historical cities, such as the Providence, Woonsocket, and Newport, have ISA over 30%. The heavily settled suburban communities have ISA between 10% and 30%. Only 17 out of 39 towns in the state have less than 10% ISA. The average ISA for the coastal towns is 14%. Because most stream quality indicators are predicted to decline when watershed ISA exceeds 10%, the results from this study serve as an alarming indicator for managing the state's watershed and coastal ecosystems. It is important to develop scientific basis for observing and modeling watershed hydrologic dynamic with emphasis on how the hydrologic processes are affected by the spatial heterogeneity of ISA, and to improve the understanding of influence of ISA on hydrologic cycle in watersheds. This developed a distributed object-oriented rainfall-runoff simulation (DORS) model that enhances capability and performance of hydrologic modeling with incorporation of high spatial resolution ISA information. With the innovative process of object-oriented spatial units, the DORS model can reduce data volume, increase computational efficiency, strengthen representation of watersheds and utilize the data in variable scales. The DORS model provides a framework to integrate remote sensing data and the derived products in different scale for the simulation of

Scene classification of high spatial resolution (HSR) images can provide data support for many practical applications, such as land planning and utilization, and it has been a crucial research topic in the remote sensing (RS) community. Recently, deep learning methods driven by massive data show the impressive ability of feature learning in the field of HSR scene classification, especially convolutional neural networks (CNNs). Although traditional CNNs achieve good classification results, it is difficult for them to effectively capture potential context relationships. The graphs have powerful capacity to represent the relevance of data, and graph-based deep learning methods can spontaneously learn intrinsic attributes contained in RS images. Inspired by the abovementioned facts, we develop a deep feature aggregation framework driven by graph convolutional network (DFAGCN) for the HSR scene classification. First, the off-the-shelf CNN pretrained on ImageNet is employed to obtain multilayer features. Second, a graph convolutional network-based model is introduced to effectively reveal patch-to-patch correlations of convolutional feature maps, and more refined features can be harvested. Finally, a weighted concatenation method is adopted to integrate multiple features (i.e., multilayer convolutional features and fully connected features) by introducing three weighting coefficients, and then a linear classifier is employed to predict semantic classes of query images. Experimental results performed on the UCM, AID, RSSCN7, and NWPU-RESISC45 data sets demonstrate that the proposed DFAGCN framework obtains more competitive performance than some state-of-the-art methods of scene classification in terms of OAs.

Content-based remote sensing (RS) image retrieval (CBRSIR) is a critical way to organize high-resolution RS (HRRS) images in the current big data era. The increasing volume of HRRS images from different satellites and sensors leads to more attention to the cross-source CSRSIR (CS-CBRSIR) problem. Due to the data drift, one crucial problem in CS-CBRSIR is the modality discrepancy. Most existing methods focus on finding a common feature space for various HRRS images to address this issue. In this space, their similarity relations can be measured directly to obtain the cross-source retrieval results straight. This way is feasible and reasonable, however, the specific information corresponding to HRRS images from different sources is always ignored, limiting retrieval performance. To overcome this limitation, we develop a new model for CS-CBRSIR in this paper named dual modality collaborative learning (DMCL). To fully explore the specific information from diverse HRRS images, DMCL first introduces ResNet50 as the feature extractor. Then, a common space mutual learning module is developed to map the specific features into a common space. Here, the modality discrepancy is reduced from the aspects of features and their distributions. Finally, to supplement the specific knowledge to the common features, we develop modality transformation and the dual-modality feature learning modules. Their function is to transmit the specific knowledge from different sources mutually and fuse the specific and common features adaptively. The comprehensive experiments are conducted on a public dataset. Compared with many existing methods, the behavior of our DMCL is stronger. These encouraging results for a public dataset indicate that the proposed DMCL is useful in CS-CBRSIR tasks.

Deep learning methods, especially deep convolutional neural networks (DCNNs), have been widely used in high-resolution remote sensing image (HRSI) semantic segmentation. In literature, most successful DCNNs are artificially designed through a large number of experiments, which often consume lots of time and depend on rich domain knowledge. Recently, neural architecture search (NAS), as a direction for automatically designing network architectures, has achieved great success in different kinds of computer vision tasks. For HRSI semantic segmentation, NAS faces two major challenges: (1) The task’s high complexity degree, which is caused by the pixel-by-pixel prediction demand in semantic segmentation, leads to a rapid expansion of the search space; (2) HRSI semantic segmentation often needs to exploit long-range dependency (i.e., a large spatial context), which means the NAS technique requires a lot of display memory in the optimization process and can be tough to converge. With the aforementioned considerations in mind, we propose a new decoupling NAS (DNAS) framework to automatically design the network architecture for HRSI semantic segmentation. In DNAS, a hierarchical search space with three levels is recommended: path-level, connection-level, and cell-level. To adapt to this hierarchical search space, we devised a new decoupling search optimization strategy to decrease the memory occupation. More specifically, the search optimization strategy consists of three stages: (1) a light super-net (i.e., the specific search space) in the path-level space is trained to get the optimal path coding; (2) we endowed the optimal path with various cross-layer connections and it is trained to obtain the connection coding; (3) the super-net, which is initialized by path coding and connection coding, is populated with kinds of concrete cell operators and the optimal cell operators are finally determined. It is worth noting that the well-designed search space can cover various network candidates and the optimization process can be done efficiently. Extensive experiments on the publicly open GID and FU datasets showed that our DNAS outperformed the state-of-the-art methods, including artificial networks and NAS methods.

Scene classification of high-resolution remote sensing images (HRRSI) is one of the most important means of land-cover classification. Deep learning techniques, especially the convolutional neural network (CNN) have been widely applied to the scene classification of HRRSI due to the advancement of graphic processing units (GPU). However, they tend to extract features from the whole images rather than discriminative regions. The visual attention mechanism can force the CNN to focus on discriminative regions, but it may suffer from the influence of intra-class diversity and repeated texture. Motivated by these problems, we propose an attention-based deep feature fusion (ADFF) framework that constitutes three parts, namely attention maps generated by Gradient-weighted Class Activation Mapping (Grad-CAM), a multiplicative fusion of deep features and the center-based cross-entropy loss function. First of all, we propose to make attention maps generated by Grad-CAM as an explicit input in order to force the network to concentrate on discriminative regions. Then, deep features derived from original images and attention maps are proposed to be fused by multiplicative fusion in order to consider both improved abilities to distinguish scenes of repeated texture and the salient regions. Finally, the center-based cross-entropy loss function that utilizes both the cross-entropy loss and center loss function is proposed to backpropagate fused features so as to reduce the effect of intra-class diversity on feature representations. The proposed ADFF architecture is tested on three benchmark datasets to show its performance in scene classification. The experiments confirm that the proposed method outperforms most competitive scene classification methods with an average overall accuracy of 94% under different training ratios.

Scene classification is an important research topic in the field of remote sensing (RS), and deep features from convolutional neural networks (CNNs) have shown good classification performance. However, a key issue is how to effectively combine context features for further improving classification accuracy. In this letter, an end-to-end framework termed deep neural network combined with context features (CFDNN) is proposed for scene classification. At first, the pretrained VGG-16 is transferred as feature extractor to obtain convolutional features. Then, two parallel modules, global average pooling (GAP) and long short-term memory (LSTM), are employed to extract global features and context features, respectively. Finally, a weighted concatenation method is introduced to combine the global and context features. As a result, the CFDNN method can adapt high spatial resolution (HSR) images with arbitrary size and obtain satisfactory classification accuracy. The experimental results on the aerial image data set (AID) demonstrate that the proposed CFDNN method has competitive classification performance compared with some state-of-the-art methods.

In this paper, we evaluate the capability of the high spatial resolution airborne Digital Airborne Imaging System (DAIS) imagery for detailed vegetation classification at the alliance level with the aid of ancillary topographic data. Image objects as minimum classification units were generated through the Fractal Net Evolution Approach (FNEA) segmentation using eCognition software. For each object, 52 features were calculated including spectral features, textures, topographic features, and geometric features. After statistically ranking the importance of these features with the classification and regression tree algorithm (CART), the most effective features for classification were used to classify the vegetation. Due to the uneven sample size for each class, we chose a non-parametric (nearest neighbor) classifier. We built a hierarchical classification scheme and selected features for each of the broadest categories to carry out the detailed classification, which significantly improved the accuracy. Pixel-based maximum likelihood classification (MLC) with comparable features was used as a benchmark in evaluating our approach. The objectbased classification approach overcame the problem of saltand-pepper effects found in classification results from traditional pixel-based approaches. The method takes advantage of the rich amount of local spatial information present in the irregularly shaped objects in an image. This classification approach was successfully tested at Point Reyes National Seashore in Northern California to create a comprehensive vegetation inventory. Computer-assisted classification of high spatial resolution remotely sensed imagery has good potential to substitute or augment the present ground-based inventory of National Park lands.

Image understanding and scene classification are keystone tasks in computer vision. The development of technologies and profusion of existing datasets open a wide room for improvement in the image classification and recognition research area. Notwithstanding the optimal performance of exiting machine learning models in image understanding and scene classification, there are still obstacles to overcome. All models are data-dependent that can only classify samples close to the training set. Moreover, these models require large data for training and learning. The first problem is solved by few-shot learning, which achieves optimal performance in object detection and classification but with a lack of eligible attention in the scene classification task. Motivated by these findings, in this paper, we introduce two models for few-shot learning in scene classification. In order to trace the behavior of those models, we also introduce two datasets (MiniSun; MiniPlaces) for image scene classification. Experimental results show that the proposed models outperform the benchmark approaches in respect of classification accuracy.

In the last decade, several high resolution remote sensing benchmark datasets have been developed and publicly released. These datasets, while diverse in design, lack the required intra-class variation for high-performing, machine-assisted visual analytics. More specifically, the disparate datasets are suitable for small, closed system evaluation; however they are not well suited for training of computer vision models that are robust in real-world, non-closed environments encountered in true remote sensing applications. To that end, a benchmark meta-dataset (MDS) was developed to facilitate the training of models for machine-assisted visual analytics. Four existing benchmark datasets were combined to build the original MDS, which excelled for training models for both classification and broad area search applications. In this work, we evaluate an enhanced version of the MDS, MDSv2, by integrating co-occurring classes of two additional recently released, publicly available challenge datasets: xView and Functional Map of the World (FMoW). The MDSv2 has 33 classes with 87,470 total samples. We investigate the utility of three neural architecture search (NAS) deep learning architectures on the MDSv2 for both classification and machine-assisted visual analytics. The NAS models trained with the MDSv2 are able to achieve an average F1 of 98.01% and a powerful 0.934 scanning precision.

Lake surface water temperature (LSWT) is an important factor of water ecological environment. As global warming, LSWT is also on the rise. Research on the main reasons of LSWT rising is the basis for controlling and improving the regional ecological environment. However, it is difficult for the existing remote sensing images to take into account the temporal and spatial resolution. Low-resolution images have a serious impact on data accuracy and even produce incorrect results. Therefore, obtaining high temporal and spatial resolution images by downscaling is of great significance to more accurately analyze the temporal and spatial characteristics of LSWT. In this article, Dianchi Lake is selected as research area, and the high spatial resolution image (Landsat) and high temporal resolution image (MODIS) are taken as data. Based on the downscaling algorithm of statistics and learning, DisTrad– super-resolution convolutional neural network (SRCNN) downscaling model is proposed, and the monthly average dataset of LSWT with 50 m resolution is constructed. The results showed 1) DisTrad–SRCNN can reflect the most distribution characteristics of LSWT (SSIMday = 0.96, PSNRday = 23.97; SSIMnight = 0.95, PSNRnight = 24.99). 2) LSWT had an overall upward trend (CRday = 0.22 °C/10a, CRnight = 0.21 °C/10a), showing a cyclical change of cold–warm–cold about 4 years. 3) The northern and lakeshore area were basically in the high temperature, and the whole lake presents a 4–5-year warm–cold–warm periodic change; the LSWT closer to the urban and residential areas and its change rate were relatively high, which indirectly reflected the serious impact of human activities on LSWT.

Based on analysis of roads features in high resolution remote sensing (RS) images, we improve the traditional algorithm for automatic edge detection and extraction and put forward a new methodology. Mathematical morphology reconstruction and boundary tracing algorithm are introduced to automatically detect and extract edges from high resolution RS images. Three factors, P factor, F factor and C factor, are defined to describe the shape characteristics of edges. K-mean cluster algorithm is then utilized to identify and extract road edges from the huge edge sets. Finally an example gives fine result of this methodology.