Multi-Label Scene Classification on Remote Sensing Imagery Using Modified Dingo Optimizer With Deep Learning

Several large-scale porphyry copper deposits (PCDs) with high economic value have been excavated in the Duolong ore district, Tibet, China. However, the high altitudes and harsh conditions in this area make traditional exploration difficult. Hydrothermal alteration minerals related to PCDs with diagnostic spectral absorption features in the visible–near-infrared–shortwave-infrared ranges can be effectively identified by remote sensing imagery. Mainly based on hyperspectral imagery supplemented by multispectral imagery and geochemical element data, the Duolong ore district was selected to conduct data-driven PCD prospectivity modelling. A total of 11 known deposits and 17 evidential layers of multisource geoscience information related to Cu mineralization constitute the input datasets of the predictive models. A deep learning convolutional neural network (CNN) model was applied to mineral prospectivity mapping, and its applicability was tested by comparison to conventional machine learning models, such as support vector machine and random forest. CNN achieves the greatest classification performance with an accuracy of 0.956. This is the first trial in Duolong to conduct mineral prospectivity mapping combined with remote imagery and geochemistry based on deep learning methods. Four metallogenic prospective sites were delineated and verified through field reconnaissance, indicating that the application of deep learning-based methods in PCD prospecting proposed in this paper is feasible by utilizing geoscience big data such as remote sensing datasets and geochemical elements.

Many objects in the naturalenvironment are generated from the background and even transformed by nature or human beings. Thus, they do not have closed and well-defined boundaries in remote sensing imagery. Recently, convolutional neural network (CNN) based object detection achieved great success in the remote sensing field. However, there is no investigation in the literature about the detection of objects with ambiguous boundaries. In this article, taking the case of the potential loess landslide detection, we designed massive experiments to evaluate convolutional neural networks for detecting objects with ambiguous boundaries in remote sensing imagery. We analyzed the evaluated methods comprehensively by comparing the performance on objects with ambiguous boundaries in remote sensing imagery with the performance on ordinary objects in visual imagery. Furthermore, drawing from these analyses, we provided a fundamental principle of object representation and a meaningful suggestion of information learning to detect objects with ambiguous boundaries. We finished this article by presenting several promising directions for detecting objects with ambiguous boundaries to facilitate and spur future research. This article would provide a significant reference and guidance to develop detectors for objects with ambiguous boundaries in remote sensing imagery.

Coastal aquaculture ponds, vital to marine aquaculture, are widely distributed along coastlines and hold significant economic and ecological value. Accurately mapping their distribution and types is essential for optimizing marine spatial resources and protecting coastal ecosystems. While deep learning has improved detection from remote sensing imagery, precise recognition still faces challenges such as large scale and shape variations, blurred boundaries in dense areas, and a lack of high-resolution fine-grained datasets. To this end, a Scale and Boundary Dynamic Awareness Network (BDA-Net) is proposed in this paper. Specifically, A Taylor-optimized Scale-adaptive Feature Interaction (TSFI) structure is designed to achieve efficient multi-scale feature adaptation for diverse pond sizes and shapes, overcoming fixed-scale Transformer limitations while preserving accuracy with lower computation via Taylor approximation. Moreover, a Frequency-guided Adaptive Boundary Convolution (FABC) is developed to resolve boundary confusion and detail loss by adaptively fusing multi-directional frequency features, enhancing contour-aligned sampling for improved fine-grained discrimination and recognition performance. Additionally, a high-resolution fine-grained dataset is constructed to fill the gap in low spatial resolution and coarse category subdivision of existing aquaculture pond datasets. Extensive experiments on this dataset and public benchmarks demonstrate that the proposed BDA-Net achieves an mIoU of 85.71%, outperforming the existing methods by over 5.45%.

In this paper, a new multi-objective (MO) fuzzy-clustering technique is developed for unsupervised land-cover classification in remote-sensing images. This technique automatically partitions the pixels in the spectral domain into several clusters. The new MO clustering algorithm uses the newly developed simulated-annealing-based multi-objective optimization technique (AMOSA) as the underlying optimization strategy. Centre-based cluster encoding is used. Each cluster is divided into several small hyperspherical sub-clusters, and the centres of all these small sub-clusters are encoded in a string to represent the whole cluster. For membership-value computation of points to different clusters, these sub-clusters are considered individually. However, for the purpose of objective-function evaluation, these sub-clusters are merged appropriately to form some variable number of whole clusters. Three objective functions, one reflecting the total compactness of the partitioning based on the Euclidean distance, one reflecting the total point-symmetrical compactness of the obtained partitioning and one reflecting the total line-symmetrical compactness of the obtained partitioning are considered here. These are optimized simultaneously using AMOSA in order to detect the appropriate number of clusters and the appropriate partitioning from remote-sensing image datasets. The point symmetry present in a particular partitioning is measured using a newly developed point-symmetry-based distance, whereas the line symmetry present in a particular partitioning is measured using a newly developed line-symmetry-based distance. Since AMOSA, as well as any other MO optimization technique, provides a set of Pareto-optimal solutions, a new method is also developed to determine a single solution from this set. Different land-cover regions in remote-sensing imagery have been classified using the proposed technique to establish its efficiency. Results are compared with those obtained by fuzzy C-means (FCM) clustering technique and a recently developed symmetry-based automatic clustering technique, the variable string length genetic point symmetry (VGAPS) technique, both qualitatively and quantitatively. Qualitative comparisons are also carried out with the segmentation results obtained by the mean-shift-based segmentation technique.

Convolutional Neural Network (CNN) is most promising for remote sensing imagery classification. However, a CNN model usually contains multiple convolutional and pooling layers for image feature extraction, which consumes lots of computational resources and makes it difficult for environmental scientists to utilize it. This study proposed the innovative Hierarchical Active Learning framework to deal with the situation when working with large satellite imagery with no need for a large number of labelled samples and expensive computational resources. The results of the study have demonstrated the value of multi-sourced data, which has opened up a new area in citizen science.

Driven by the urgent demand for flood monitoring, water resource management and environmental protection, water-body detection in remote sensing imagery has attracted increasing research attention. Deep semantic segmentation networks (DSSNs) have gradually become the mainstream technology used for remote sensing image water-body detection, but two vital problems remain. One problem is that the traditional structure of DSSNs does not consider multiscale and multishape characteristics of water bodies. Another problem is that a large amount of unlabeled data is not fully utilized during the training process, but the unlabeled data often contain meaningful supervision information. In this paper, we propose a novel multiscale residual network (MSResNet) that uses self-supervised learning (SSL) for water-body detection. More specifically, our well-designed MSResNet distinguishes water bodies with different scales and shapes and helps retain the detailed boundaries of water bodies. In addition, the optimization of MSResNet with our SSL strategy can improve the stability and universality of the method, and the presented SSL approach can be flexibly extended to practical applications. Extensive experiments on two publicly open datasets, including the 2020 Gaofen Challenge water-body segmentation dataset and the GID dataset, demonstrate that our MSResNet can obviously outperform state-of-the-art deep learning backbones and that our SSL strategy can further improve the water-body detection performance.

Interactive segmentation (IS) using minimal prompts like points and bounding boxes facilitates rapid image annotation, which is crucial for enhancing data-driven deep learning methods. Traditional IS methods, however, process only one target per interaction, leading to inefficiency when annotating multiple identical-class objects in remote sensing imagery (RSI). To address this issue, we present a new task—identical-class object detection (ICOD) for rapid IS in RSI. This task aims to only identify and detect all identical-class targets within an image, guided by a specific category target in the image with its mask. For this task, we propose an ICOD network (ICODet) with a two-stage object detection framework, which consists of a backbone, feature similarity analysis module (S3QFM), and an identical-class object detector. In particular, the S3QFM analyzes feature similarities from images and support objects at both feature-space and semantic levels, generating similarity maps. These maps are processed by a region proposal network (RPN) to extract target-level features, which are then refined through a simple feature comparison module and classified to precisely identify identical-class targets. To evaluate the effectiveness of this method, we construct two datasets for the ICOD task: one containing a diverse set of buildings and another containing multicategory RSI objects. Experimental results show that our method outperforms the compared methods on both datasets. This research introduces a new method for rapid IS of RSI and advances the development of fast interaction modes, offering significant practical value for data production and fundamental applications in the remote sensing community.

In recent years, remote-sensing (RS) technologies have been used together with image processing and traditional techniques in various disaster-related works. Among these is detecting building damage from orthophoto imagery that was inflicted by earthquakes. Automatic and visual techniques are considered as typical methods to produce building damage maps using RS images. The visual technique, however, is time-consuming due to manual sampling. The automatic method is able to detect the damaged building by extracting the defect features. However, various design methods and widely changing real-world conditions, such as shadow and light changes, cause challenges to the extensive appointing of automatic methods. As a potential solution for such challenges, this research proposes the adaption of deep learning (DL), specifically convolutional neural networks (CNN), which has a high ability to learn features automatically, to identify damaged buildings from pre- and post-event RS imageries. Since RS data revolves around imagery, CNNs can arguably be most effective at automatically discovering relevant features, avoiding the need for feature engineering based on expert knowledge. In this work, we focus on RS imageries from orthophoto imageries for damaged-building detection, specifically for (i) background, (ii) no damage, (iii) minor damage, and (iv) debris classifications. The gist is to uncover the CNN architecture that will work best for this purpose. To this end, three CNN models, namely the twin model, fusion model, and composite model, are applied to the pre- and post-orthophoto imageries collected from the 2016 Kumamoto earthquake, Japan. The robustness of the models was evaluated using four evaluation metrics, namely overall accuracy (OA), producer accuracy (PA), user accuracy (UA), and F1 score. According to the obtained results, the twin model achieved higher accuracy (OA = 76.86%; F1 score = 0.761) compare to the fusion model (OA = 72.27%; F1 score = 0.714) and composite (OA = 69.24%; F1 score = 0.682) models.

An understanding of environmental variability (stability/instability) is important to support operational planning of expeditionary warfare and littoral operations, as well as for preparing the Recognized Environmental Picture (REP). Specifically, the identification of environmentally stable/unstable areas helps the planning of maritime operations, increasing their likelihood of success. The purpose of the paper is to describe a methodology to form and interpret an initial spatial-temporal variability characterization of maritime areas from Remote Sensing (RS) and Numerical Ocean Model (NOM) data. As a case study, the analysis of the sea surface tem- perature (SST) in the Black Sea from historical time-series of RS imagery and NOM data is considered. The results of the analysis are validated with in situ measurements from moorings. Identification of gaps of geospatial information is also done in this study. The analysis is focused on monthly spatial-temporal variability of the SST, generating stability maps displaying the geospatial distribution of environmentally stable/unstable areas along a year. The results show how the proposed methodology captures the temporal variability of the SST in the Black Sea, being compared with in situ measurements, and provides useful information for the identification of environmentally stable/unstable areas. The results show a general agreement in the variability with both RS and NOM data, when RS imagery may be used for the present analysis, i.e. when low cloud coverage is given. This paper demonstrates that when RS imagery gaps are not negligible (e.g. due to high cloud occurrence in winter season), these gaps could be filled with NOM data.

Validation Challenges in Large-Scale Tree Crown Segmentations from Remote Sensing Imagery Using Deep Learning: A Case Study in Germany

Abstract. Evolutionary computation is used for improved information extraction from high-resolution satellite imagery. The utilization of evolutionary computation is based on stochastic selection of input parameters often defined in a trial-and-error approach. However, exploration of optimal input parameters can yield improved candidate solutions while requiring reduced computation resources. In this study, the design and implementation of a system that investigates the optimal input parameters was researched in the problem of feature extraction from remotely sensed imagery. The two primary assessment criteria were the highest fitness value and the overall computational time. The parameters explored include the population size and the percentage and order of mutation and crossover. The proposed system has two major subsystems; (i) data preparation: the generation of random candidate solutions; and (ii) data processing: evolutionary process based on genetic programming, which is used to spectrally distinguish the features of interest from the remaining image background of remote sensed imagery. The results demonstrate that the optimal generation number is around 1500, the optimal percentage of mutation and crossover ranges from 35% to 40% and 5% to 0%, respectively. Based on our findings the sequence that yielded better results was mutation over crossover. These findings are conducive to improving the efficacy of utilizing genetic programming for feature extraction from remotely sensed imagery.

Marine oil spill incidents are one of the major global marine pollution issues, which pose significant threats to ocean ecosystems. However, traditional monitoring methods often suffer from time delays, high costs, and limited real-time capability, making them inadequate for timely and large-scale oil spill detection. With the development of remote sensing (RS) technology and artificial intelligence (AI) methods, as well as the increasing frequency of marine oil spill accidents, plenty of AI-based methods using RS imagery have been proposed for more efficient and accurate oil spill monitoring. This review presents a comprehensive and systematic overview of recent progress in marine oil spill analysis using RS imagery, emphasizing the integration of AI methods across three key tasks: detection, classification, and thickness estimation. Specifically, we first introduce the main types of RS data and discuss the significance of publicly available datasets, which can facilitate method validation and model comparison. Second, we briefly review the application of RS imagery from different sensors in oil spill detection, highlighting the strengths of various spectral and polarimetric methods. Third, we summarize advances in oil spill classification, including AI-based methods that enable differentiation between mineral oil, biogenic films, and various emulsified oils. Fourth, we discuss emerging techniques for oil spill thickness estimation. Finally, we analyze the challenges of existing methods and future directions, including the need for real-time monitoring, the integration of multi-source RS data, and the development of robust models that can generalize across different environmental conditions. This review adopts a comprehensive perspective from both AI methods and RS technology, provides a systematic overview of recent advancements, identifies critical gaps in current methodologies, and serves as a valuable reference for researchers and practitioners working on oil spill monitoring.

Portraying urban functional zones provides useful insights into understanding complex urban systems and establishing rational urban planning. Although several studies have confirmed the efficacy of remote sensing imagery in urban studies, coupling remote sensing and new human sensing data like mobile phone positioning data to identify urban functional zones has still not been investigated. In this study, a new framework integrating remote sensing imagery and mobile phone positioning data was developed to analyze urban functional zones with landscape and human activity metrics. Landscapes metrics were calculated based on land cover from remote sensing images. Human activities were extracted from massive mobile phone positioning data. By integrating them, urban functional zones (urban center, sub-center, suburbs, urban buffer, transit region and ecological area) were identified by a hierarchical clustering. Finally, gradient analysis in three typical transects was conducted to investigate the pattern of landscapes and human activities. Taking Shenzhen, China, as an example, the conducted experiment shows that the pattern of landscapes and human activities in the urban functional zones in Shenzhen does not totally conform to the classical urban theories. It demonstrates that the fusion of remote sensing imagery and human sensing data can characterize the complex urban spatial structure in Shenzhen well. Urban functional zones have the potential to act as bridges between the urban structure, human activity and urban planning policy, providing scientific support for rational urban planning and sustainable urban development policymaking.

Boosting ICD multi-label classification of health records with contextual embeddings and label-granularity

Prediction of the stage of cancer plays an important role in planning the course of treatment and has been largely reliant on imaging tools which do not capture molecular events that cause cancer progression. Gene-expression data-based analyses are able to identify these events, allowing RNA-sequence and microarray cancer data to be used for cancer analyses. Breast cancer is the most common cancer worldwide, and is classified into four stages - stages 1, 2, 3, and 4 [2]. While machine learning models have previously been explored to perform stage classification with limited success, multi-class stage classification has not had significant progress. There is a need for improved multi-class classification models, such as by investigating deep learning models. Gene-expression-based cancer data is characterised by the small size of available datasets, class imbalance, and high dimensionality. Class balancing methods must be applied to the dataset. Since all the genes are not necessary for stage prediction, retaining only the necessary genes can improve classification accuracy. The breast cancer samples are to be classified into 4 classes of stages 1 to 4. Invasive ductal carcinoma breast cancer samples are obtained from The Cancer Genome Atlas (TCGA) and Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) datasets and combined. Two class balancing techniques are explored, synthetic minority oversampling technique (SMOTE) and SMOTE followed by random undersampling. A hybrid feature selection pipeline is proposed, with three pipelines explored involving combinations of filter and embedded feature selection methods: Pipeline 1 - minimum-redundancy maximum-relevancy (mRMR) and correlation feature selection (CFS), Pipeline 2 - mRMR, mutual information (MI) and CFS, and Pipeline 3 - mRMR and support vector machine-recursive feature elimination (SVM-RFE). The classification is done using deep learning models, namely deep neural network, convolutional neural network, recurrent neural network, a modified deep neural network, and an AutoKeras generated model. Classification performance post class-balancing and various feature selection techniques show marked improvement over classification prior to feature selection. The best multiclass classification was found to be by a deep neural network post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with a Cohen-Kappa score of 0.303 and a classification accuracy of 53.1%. For binary classification into early and late-stage cancer, the best performance is obtained by a modified deep neural network (DNN) post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with an accuracy of 81.0% and a Cohen-Kappa score (CKS) of 0.280. This pipeline also showed improved multiclass classification performance on neuroblastoma cancer data, with a best area under the receiver operating characteristic (auROC) curve score of 0.872, as compared to 0.71 obtained in previous work, an improvement of 22.81%. The results and analysis reveal that feature selection techniques play a vital role in gene-expression data-based classification, and the proposed hybrid feature selection pipeline improves classification performance. Multi-class classification is possible using deep learning models, though further improvement particularly in late-stage classification is necessary and should be explored further.