A Spatial–Spectral Prototypical Network for Hyperspectral Remote Sensing Image

Abnormal target detection in hyperspectral remote sensing image is one of the hotspots in image research. The image noise generated in the detection process will lead to the decline of the quality of hyperspectral remote sensing image. In view of this, this paper proposes an abnormal target detection method of hyperspectral remote sensing image based on the convolution neural network. Firstly, the deep residual learning network model has been used to remove the noise in hyperspectral remote sensing image. Secondly, the spatial and spectral features of hyperspectral remote sensing images were used to optimize the clustering dictionary, and then the image segmentation containing target information is completed. Finally, the image was input into the deep convolution neural network with a dual classifier, and the network detects the abnormal target in the image. The test results of this algorithm show that the structural similarity of the denoised image is higher than 0.86, which shows that this method has good noise reduction performance, image details will not damage, segmentation effect is good, and it can obtain high-definition target image information and accurately detect abnormal targets in the image.

To effectively utilize the rich spectral information of hyperspectral remote sensing images (HRSIs), the fractional Fourier transform (FRFT) feature of HRSIs is proposed to reflect the time-domain and frequency-domain characteristics of a spectral pixel simultaneously, and an FRFT order selection criterion is also proposed based on maximizing separability. Firstly, FRFT is applied to the spectral pixels, and the amplitude spectrum is taken as the FRFT feature of HRSIs. The FRFT feature is mixed with the pixel spectral to form the presented spectral and fractional Fourier transform mixed feature (SF2MF), which contains time–frequency mixing information and spectral information of pixels. K-nearest neighbor, logistic regression, and random forest classifiers are used to verify the superiority of the proposed feature. A 1-dimensional convolutional neural network (1D-CNN) and a two-branch CNN network (Two-CNNSF2MF-Spa) are designed to extract the depth SF2MF feature and the SF2MF-spatial joint feature, respectively. Moreover, to compensate for the defect that CNN cannot effectively capture the long-range features of spectral pixels, a long short-term memory (LSTM) network is introduced to be combined with CNN to form a two-branch network C-CLSTMSF2MF for extracting deeper and more efficient fusion features. A 3D-CNNSF2MF model is designed, which firstly performs the principal component analysis on the spa-SF2MF cube containing spatial information and then feeds it into the 3-dimensional convolutional neural network 3D-CNNSF2MF to extract the SF2MF-spatial joint feature effectively. The experimental results of three real HRSIs show that the presented mixed feature SF2MF can effectively improve classification accuracy.

The classification of hyperspectral remote sensing images (HSI) is an important task in the processing and application of hyperspectral images. Convolutional neural network (CNN) has significant advantages in the extraction and fusion of hyperspectral image spectrum and spatial features, so it has become a common method in the field of HSI classification. However, the spectral domain feature redundancy of hyperspectral images is very high, and the spatial domain feature structure is complex, which makes CNN more time-consuming and higher memory requirements in the feature extraction process. Based on this problem, lightweight networks with fewer parameters have gradually become the main application method in the field of HSI classification. In order to reduce the computational complexity of deep feature extraction and take into account the shallow features, a high-order nonlinear Ghost module is proposed on the basis of the original Ghost linear transformation module. Furthermore, in view of the independent characteristics of each dimension of HSI, a Trip-GhostNet is proposed, which simultaneously extracts and fuses features from three dimensions in a lightweight manner. According to the distribution characteristics of HSI, the influence of attention embedding methods in the high-order Ghost module of each branch on the feature extraction is compared and analyzed. The results show that the proposed model can reduce model calculations and improve classification accuracy, and is suitable for HSI classification problems.

Hyperspectral remote sensing image (HSI) consists of hundreds of bands that contain rich space, radiation and spectral information. The high-dimensional data can also lead to the curse of dimensionality problem making it difficult to be used effectively. In this paper, we proposed a manifold learning algorithm to reduce the dimensionality for HSI data. For high dimensional datasets with continuous variables, it is often the case that the data points are arranged along with low dimensional structures, named manifolds, in the high dimensional space. Manifold learning aims to identifying those special low dimensional structures for subsequent usage such as classification or regression. However, many manifold learning algorithms perform an eigenvector analysis on a data similarity matrix whose size is N×N, where N is the number of data points. The memory complexity of the analysis is at least O(N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) that is not feasible for a regular computer to compute or storage for very large datasets. To solve this problem, we used statistical sampling methods to sample a subset of data points as landmarks. A skeleton of the manifold was then identified based on the landmarks. The remaining data points were then inserted into the skeleton by Locally Linear Embedding (LLE). We tested our algorithm on AVIRIS Salinas-A data set. The experimental results showed that the HSI dataset could be reduced to a lower-dimensional space for land use classification with good performance, and the main structure was preserved well.

ABSTRACTThis study introduces the attention mechanism in hyperspectral remote sensing image (HSI) classification which can strengthen the information provided by important features, and weaken the non-essential information. We introduced the Squeeze-and-Excitation (SE) block embedded in three-dimensional densely connected convolutional network (3D-DenseNet) to form 3D-SE-DenseNet for HSI classifications. This model can learn a powerful network with low training costs and fast convergence speed, and avoids overfitting on small sample datasets. Two different 3D-SE-DenseNet models of 3D-SE-DenseNet and 3D-SE-DenseNet-BC were set up. Results from experiments show that the 3D-SE-DenseNet performs well on the Indian Pines, Pavia University, Botswana, and Kennedy Space Centre datasets.

Graph neural network has excellent performance in obtaining the similarity relationship of samples, so it has been widely used in computer vision. But hyperspectral remote sensing image (HSI) has some problems such as data redundancy, noise, lack of labeled samples, and insufficient utilization of spatial information. These problems affect the accuracy of HSI classification using graph neural networks. To solve the above problems, this paper proposes graph-based semi-supervised learning with weighted features for HSI classification. The method proposed in this paper first uses the stacked autoencoder network to extract features, which is used to remove the redundancy of HSI data. Then, the similarity attenuation coefficient is introduced to improve the original feature weighting scheme. In this way, the contribution difference of adjacent pixels to the center pixel is reflected. Finally, to obtain more generalized spectral features, a shallow feature extraction mechanism is added to the stacked autoencoder network. And features that have good generalization can solve the problem of the lack of labeled samples. The experiment on three different types of datasets demonstrates that the proposed method in this paper can get better classification performance in the case of the scarcity of labeled samples than other classification methods.

Research on spatial information is a hot topic in hyperspectral remote-sensing image (HSI) exploitation. This paper defines a spatial correlation filter to study the spatial variation characteristics and intensity of regionalized variables. Unlike existing spatial information expression methods based on theoretical model, the novel filter takes multi-scale and multi-directional spatial structure features into account and visualizes them, which is model-free and more comprehensive. Further, SCFB model constructed by the filter banks is proposed for pixel-wise classification of HSI and performs well when replacing the classifier component with multiple machine-learning algorithms. Accurate classification on two classic hyperspectral datasets with approximately 3% training samples and on one large dataset with 0.544% training samples indicate that the model is promising in small samples learning. In particular, supervised linear discriminant analysis (LDA) used in the feature subspace optimization part of the model significantly outperforms the other two spectral strategies. Most notably, the model is not only superior to the traditional machine-learning methods considering spectral information only, but also has advantages in accuracy, the number of learnable kernel parameters and time-consuming compared with equal-layer convolution neural network.

Hyperspectral remote sensing (RS) involves capturing and analyzing data across numerous, narrow spectral bands in the electromagnetic spectrum. With this technology, it is possible to identify what has been on the Earth surface with unprecedented depth that cannot be done with the mere multispectral images. A Hyperspectral sensor is able to capture hundreds of bands of data, and is therefore able to distinguish between very small differences in surface materials that we cannot even see or conventional multispectral sensors. Hyperspectral remote sensing is widely used in applications like agriculture, environmental monitoring, geology, and defence. To overcome the limitations of current approaches, new strategies are needed. In this manuscript Optimized Dilated Convolutional Neural Network with Quantum Self-Attention Based on Hyper Spectral Remote Sensing Image Classification (DCNN-QSA-PDOA) is proposed. The Indian Pine dataset is where the data is first gathered. The multidirectional shearlet transform based feature extraction (MDST) model is then utilized to extract the hyperspectral image (HSI) features. Following the completion of the feature extraction procedure, the Dilated Convolutional Neural Network with Quantum Self-Attention (DCNN-QSA) is trained on the feature extracted images and used to classify the category information. Here, (PDOA) is utilized for tuning hyper parameters of DCNN-QSA. The proposed DCNN-QSA-PDOA method provides 37.23%, 18.74% and 24.84% higher accuracy compared with existing method like Classifying images from hyper spectral RS using a DCNN-HIS, Dense residual three-dimensional convolutional neural networks are used for hyper spectral RS picture classification (DR-3DCNN-HIS), Fully convolutional neural networks are used for the categorization of hyper spectral remote sensing pictures (CNN-HIS) respectively.

Hyperspectral band selection (BS) is an important technique to reduce data dimensionality for the classification applications of hyperspectral remote sensing images (HSIs). Recently, searching-based BS methods have received increasing attention for their ability to select the best subset of bands while preserving the essential information of the original data. However, existing searching-based BS methods neglect effective exploitation of the spatial and spectral prior information inherent in the data, thus limiting their performance. To address this problem, in this study, a novel unsupervised BS method called Spectral–Spatial Iterative Greedy Algorithm (SSIGA) is proposed. Specifically, to facilitate efficient local search using spectral information, SSIGA conducts clustering on all the bands by employing a K-means clustering method with balanced cluster size constraints and constructs a K-nearest neighbor graph for each cluster. Based on the nearest neighbor graphs, SSIGA can effectively explore the neighborhood solutions in local search. In addition, to efficiently evaluate the discriminability and information redundancy of the solution given by SSIGA using the spatial and spectral information of HSIs, we designed an effective objective function for SSIGA. The value of the objective function is derived by calculating the Fisher score for each band in the solution based on the results of the superpixel segmentation performed on the target HSI, as well as by computing the average information entropy and mutual information of the bands in the solution. Experimental results on three publicly available real HSI datasets demonstrate that the SSIG algorithm achieves superior performance compared to several state-of-the-art methods.

Eggs are important nutritional foods in people's daily life. They are rich in protein and vitamins, as well as a variety of minerals like iron, calcium and phosphorus. Therefore, monitoring egg quality is always a focus problem that people are concerned about. This article breaks through traditional ideas and methods, proposes using hyperspectral remote sensing images to obtain the spectral characteristics of boiled yolk, and establishes the analysis model of egg quality. Through a large number of experiments on different types of eggs, the experimental results are effective, and this shows that the method proposed in this paper is feasible. It provides new ideas and methods for the examination of egg quality and the establishment of traceability system.

This paper explores the problem of recovering the discriminative representation of a hyperspectral remote sensing image (HRSI), which suffers from spectral variations, to boost its classification accuracy. To tackle this challenge, we propose a new method, namely local-global balanced low-rank approximation (GLB-LRA), which can increase the similarity between pixels belonging to an identical category while promoting the discriminability between pixels of different categories. Specifically, by taking advantage of the particular structural spatial information of HRSIs, we exploit the low-rankness of an HRSI robustly in both spatial and spectral domains from the perspective of local and global balance. We mathematically formulate GLB-LRA as an explicit optimization problem and propose an iterative algorithm to solve it efficiently. Experimental results over three commonly-used benchmark datasets demonstrate the significant superiority of our method over state-of-the-art methods.

Multimodal emotion recognition has become a hot topic in human-computer interaction and intelligent healthcare fields. However, combining information from different human different modalities for emotion computation is still challenging. In this paper, we propose a three-dimensional convolutional recurrent neural network model (referred to as 3FACRNN network) based on multimodal fusion and attention mechanism. The 3FACRNN network model consists of a visual network and an EEG network. The visual network is composed of a cascaded convolutional neural network-time convolutional network (CNN-TCN). In the EEG network, the 3D feature building module was added to integrate band information, spatial information and temporal information of the EEG signal, and the band attention and self-attention modules were added to the convolutional recurrent neural network (CRNN). The former explores the effect of different frequency bands on network recognition performance, while the latter is to obtain the intrinsic similarity of different EEG samples. To investigate the effect of different frequency bands on the experiment, we obtained the average attention mask for all subjects in different frequency bands. The distribution of the attention masks across the different frequency bands suggests that signals more relevant to human emotions may be active in the high frequency bands γ (31-50 Hz). Finally, we try to use the multi-task loss function Lc to force the approximation of the intermediate feature vectors of the visual and EEG modalities, with the aim of using the knowledge of the visual modalities to improve the performance of the EEG network model. The mean recognition accuracy and standard deviation of the proposed method on the two multimodal sentiment datasets DEAP and MAHNOB-HCI (arousal, valence) were 96.75 ± 1.75, 96.86 ± 1.33; 97.55 ± 1.51, 98.37 ± 1.07, better than those of the state-of-the-art multimodal recognition approaches. The experimental results show that starting from the multimodal information, the facial video frames and electroencephalogram (EEG) signals of the subjects are used as inputs to the emotion recognition network, which can enhance the stability of the emotion network and improve the recognition accuracy of the emotion network. In addition, in future work, we will try to utilize sparse matrix methods and deep convolutional networks to improve the performance of multimodal emotion networks.

Abstract. Mud volcanoes are geological formations resulting from the expulsion of mud, gases, and fluids from deep underground. Monitoring these formations provides critical insights into subsurface processes and geological hazards. This study focuses on detecting recent mud extrusions in mud volcano environments using high-resolution aerial imagery acquired by unmanned aerial vehicles (UAVs). Using UAV-based surveys instead of satellite imagery, we obtain finer spatial detail suitable for identifying subtle textural and chromatic variations in relatively small sites. A binary image classification pipeline was developed to distinguish recent mud from non-mud areas. Traditional machine learning algorithms, including Support Vector Machine (SVM), Random Forest, and Extreme Gradient Boosting (XGBoost), were compared with deep learning architectures such as Convolutional Neural Networks (CNNs), both leveraging transfer learning and custom models. Traditional algorithms rely on handcrafted features, while CNNs learn hierarchical representations directly from raw data. Feature extraction methods were selected based on their ability to distinguish between the two designated classes effectively. To enhance model robustness and generalization, a designed augmentation pipeline was applied before each training epoch or cross-validation fold. This strategy introduced controlled and random variations to simulate real-world imaging conditions, such as varying viewpoints and lighting, ensuring the models generalization, moreover it also minimized data leakage by presenting distinct image variations throughout training. CNNs achieved the highest accuracy, outperforming traditional algorithms and demonstrating the advantages of combining deep learning with effective data augmentation. These findings underscore the potential of CNNs for accurate and efficient monitoring of dynamic geological environments.

Aiming at the problem of “curse of dimensionality” in hyperspectral image (HSI) classification, this paper proposes an HSI feature extraction algorithm combining spatial-spectral information. The proposed algorithm makes full use of the spatial information and spectral information of HSI. First, the image is reconstructed by introducing the principle of spatial consistency, reduces the redundancy of local information, and incorporates spatial structure information into the spectral feature set automatically. Then, the traditional PCA-LPP manifold learning algorithm is improved to extract spatial-spectral information based on the reconstructed image. The Parzen window density estimation method is used to adapt the neighborhood size, the local spatial features of the image are extracted. At the same time, the neighbor graph is constructed based on the spatial-spectral distance metric, so that the similarity measure between data points is better realized. Finally, the extracted spatial-spectral features are input into a Support Vector Machine (SVM) for HSI classification. Experimental results on Salinas and Pavia U datasets show that the algorithm has better classification accuracy than traditional algorithms.

Due to the benefits of convolutional neural networks (CNNs) in image classification, they have been extensively used in the computerized classification and focus of crop pests. The intention of the current find out about is to advance a deep convolutional neural network to mechanically identify 14 species of tea pests that possess symmetry properties. (1) As there are not enough tea pests images in the network to train the deep convolutional neural network, we proposes to classify tea pests images by fine-tuning the VGGNET-16 deep convolutional neural network. (2) Through comparison with traditional machine learning algorithms Support Vector Machine (SVM) and Multi-Layer Perceptron (MLP), the performance of our method is evaluated (3) The three methods can identify tea tree pests well: the proposed convolutional neural network classification has accuracy up to 97.75%, while MLP and SVM have accuracies of 76.07% and 68.81%, respectively. Our proposed method performs the best of the assessed recognition algorithms. The experimental results also show that the fine-tuning method is a very powerful and efficient tool for small datasets in practical problems.