Extended Spectral to Visible Comparison Based on Spectral Band Selection Method for Robust Face Recognition

Pine wilt disease (PWD) can cause destructive death in many species of pine trees within a short period. The recognition of infected pine trees in unmanned aerial vehicle (UAV) forest images is a key technology for automatic monitoring and early warning of pests. This paper collected UAV visible and multispectral images of Korean pines (Pinus koraiensis) and Chinese pines (P. tabulaeformis) infected by PWD and divided the PWD infection into early, middle, and late stages. With the open-source annotation tool, LabelImg, we labeled the category of infected pine trees at each stage. After coordinate-correction preprocessing of the ground truth, the Korean pine and Chinese pine datasets were established. As a means of detecting infected pine trees of PWD and determining different infection stages, a multi-band image-fusion infected pine tree detector (MFTD) based on deep learning was proposed. Firstly, the Halfway Fusion mode was adopted to fuse the network based on four YOLOv5 variants. Simultaneously, the Backbone network was initially designed as a dual branching network that includes visible and multispectral subnets. Moreover, the features of visible and multispectral images were extracted. To fully utilize the features of visible and multispectral images, a multi-band feature fusion transformer (MFFT) with a multi-head attention mechanism and a feed-forward network was constructed to enhance the information correlation between visible and multispectral feature maps. Finally, following the MFFT module, the two feature maps were fused and input into Neck and Head to predict the categories and positions of infected pine trees. The best-performing MFTD model achieved the highest detection accuracy with mean average precision values (mAP@50) of 88.5% and 86.8% on Korean pine and Chinese pine datasets, respectively, which improved by 8.6% and 10.8% compared to the original YOLOv5 models trained only with visible images. In addition, the average precision values (AP@50) are 87.2%, 93.5%, and 84.8% for early, middle, and late stages on the KP dataset and 81.2%, 92.9%, and 86.2% on the CP dataset. Furthermore, the largest improvement is observed in the early stage with 14.3% and 11.6%, respectively. The results show that MFTD can accurately detect the infected pine trees, especially those at the early stage, and improve the early warning ability of PWD.

One way to identify human faces is by using facial features or facial patterns based on visible images. However, this approach depends on the lighting conditions. In a poor illumination environment, it is difficult to recognize the face images. Thermal images do not rely on the illumination condition; thus can overcome this problem. Unlike the visible image, the thermal images lack visual information. Therefore, matching between the thermal and visible images (cross-spectral matching) is vital in face recognition. This paper proposed a phase correlation method, namely phase-only correlation (POC) and band-limited POC (BLPOC), for a cross-spectral matching of face images. First, the Fourier transform was applied to face images in both spectral domains, and the phase correlation between the two images was calculated. The experimental results showed that, from 100 visible and thermal image (acquired by FLIR C2 camera) pairs, the BLPOC resulted in significant correlation values compared to POC.

A perceptual-based multiresolution image fusion technique is demonstrated using the Airborne Visible and Infrared Imaging Spectrometer (AVIRIS) hyperspectral sensor data. The AVIRIS sensor, which simultaneously collects information in 224 spectral bands that range from 0.4 to 2.5 μm in approximately 10-nm increments, produces 224 images, each representing a single spectral band. The fusion algorithm consists of three stages. First, a Daubechies orthogonal wavelet basis set is used to perform a multiresolution decomposition of each spectral image. Next, the coefficients from each image are combined using a perceptual-based weighting. The weighting of each coefficient, from a given spectral band image, is determined by the spatial-frequency response (contrast sensitivity) of the human visual system. The spectral image with the higher saliency value, where saliency is based on a perceptual energy, will receive the larger weight. Finally, the fused coefficients are used for reconstruction to obtain the fused image. The image fusion algorithm is analyzed using test images with known image characteristics and image data from the AVIRIS hyperspectral sensor. By analyzing the signal-to-noise ratios and visual aesthetics of the fused images, contrast-sensitivity-based fusion is shown to provide excellent fusion results and to outperform previous fusion methods.

Different crops, as well as the same crop at different growth stages, display distinct spectral and spatial characteristics in hyperspectral images (HSIs) due to variations in their chemical composition and structural features. However, the narrow bandwidth and closely spaced spectral channels of HSIs result in significant data redundancy, posing challenges to crop identification and classification. Therefore, the dimensionality reduction in HSIs is crucial. Band selection as a widely used method for reducing dimensionality has been extensively applied in research on crop identification and mapping. In this paper, a crop superpixel-based affinity propagation (CS-AP) band selection method is proposed for crop identification and mapping in agriculture using HSIs. The approach begins by gathering crop superpixels; then, a spectral band selection criterion is developed by analyzing the variations in the spectral and spatial characteristics of crop superpixels. Finally, crop identification bands are determined through an efficient clustering approach, AP. Two typical agricultural hyperspectral data sets, the Salinas Valley data set and the Indian Pines data set, are selected for validation, each containing 16 crop classes, respectively. The experimental results show that the proposed CS-AP method achieves a mapping accuracy of 92.4% for the Salinas Valley data set and 88.6% for the Indian Pines data set. When compared to using all bands, two unsupervised band selection techniques, and three semi-supervised band selection techniques, the proposed method outperforms others with an improvement of 3.1% and 4.3% for the Salinas Valley and Indian Pines data sets, respectively. Indicate that the proposed CS-AP method achieves superior mapping accuracy by selecting fewer bands with greater crop identification capability compared to the other band selection methods. This research’s significant results demonstrate the potential of this approach in precision agriculture, offering a more cost-effective and timely solution for large-scale crop mapping and monitoring in the future.

Hyperspectral imagery consists of hundreds of spectra or bands whose intensity is measured at various wavelength. Fusing the multiple spectral bands can provide more potential to differentiate between natural and man-made objects, and significantly improve the capability of target detection and classification. Spectral band or wavelength selection is one of the fundamental problems in hyperspectral data fusion. It is one instance of the classical optimal subset selection problem, which is known to be computationally hard. In this paper, we propose a new information-based band selection method for hyperspectral image fusion, which uses an adaptive measurement of mutual information (MI). As derived from the concept of entropy, MI measures the statistical dependence between two random variables and therefore can be used to evaluate the relative utility of each band to classification. Experiments on the AVIRIS dataset show that the method effectively identifies redundant spectral bands. Removing 15% of the total bands increases accuracy by 1.76% relative to performance on all bands, whereas removing 45% of the bands gives only 1.34% loss of accuracy.

Band selection methods often suppose normal distribution or, at least, a significant number of samples per class to compute statistical parameters. In this paper, we propose a band selection technique that needs very few training samples per class to be effective. To take into account the spectral variability inside the classes and to be independent on statistic parameters, we propose to use a criterion which measures the separability between two classes. This criterion is based on the extension of Spectral Angle Mapper (SAM) to a SAM within- and between-classes. The proposed selection method consists in eliminating spectral bands from the original set, thanks to the previous criterion that check the increase of separability measure between classes on the remaining bands subset. We use a stochastic algorithm to choose, at each step, which band to eliminate. We proceed by successive elimination until we reach the number of desired bands or the maximum of the criterion. This top-down method allows taking simultaneously into account all the interesting bands during the whole process, instead of selecting them one by one. The method provides, at the end, the selected spectral bands for a pair of classes. We expand this two-class selection technique to the multiclass band selection. We improve the method by adding a pre-selection of interesting bands considering a measure of a spectral signal on noise ratio. Some examples are given to show the effectiveness of the method: as one main application is classification, we compare the results of classification achieved after the data reduction made by different methods. We check the efficiency according to the number of training samples.

Part 3 of this Special Section on High-Performance Computing in Applied Remote Sensing continues to present the state-of-the-art research in incorporating high-performance computing facilities and algorithms for effective and efficient remote sensing applications. “Land cover classification in multispectral imagery using clustering of sparse approximations over learned feature dictionaries” by Moody et al. presents results from an ongoing effort to extend neuromimetic machine vision algorithms to multispectral data using adaptive signal processing combined with compressive sensing and machine learning techniques. “Graphics processor unit accelerated finite-difference time domain method for electromagnetic scattering from onedimensional large scale rough soil surface at low grazing incidence” by Jia et al. applies the Compute Unified Device Architecture (CUDA) technology to compute an electromagnetic scattering problem on GPUs and achieves significant speedup factors. “High-performance meshing processing of remote sensing data on large displays” by Hsieh et al. presents a technique that visualizes LIDAR point clouds on a tiled display wall, termed highly interactive-parallelized display (HIPerDisplay), which has twenty 24-inch liquid-crystal displays with a total resolution of 46 Mpixels. The results also showed that the HIPerDisplay offered superior performance for the processing of large LIDAR datasets. “Real-time implementation of optimized maximum noise fraction transform for feature extraction of hyperspectral images” by Wu et al. presents a parallel implementation of the optimized maximum noise fraction (G-OMNF) transform algorithm for feature extraction of hyperspectral images on commodity graphics processing units (GPUs). The proposed approach explored the algorithm data-level concurrency and optimized the computing flow. “Hyperspectral band selection based on parallel particle swarm optimization and impurity function band prioritization schemes” by Chang et al. presents a framework for band selection in hyperspectral imagery that uses two techniques, referred to as particle swarm optimization (PSO) band selection and the impurity function band prioritization (IFBP) method. The experimental results demonstrate that the proposed PPSO/IFBP band selection method can not only improve computational speed but also offer a satisfactory classification performance. “Graphics processing unit implementation of the maximum likelihood solution to the inverse problem for retrieval of geophysical parameters from high-resolution sounder data” by Wei et al. explores the use of GPU in speedup of the maximum likelihood solution to an illposed retrieval problem. For the infrared atmospheric sounding interferometer high-resolution sounder having 8641 channels, the use of GPU on maximum likelihood estimation shows a promising speedup of 1986× compared to a single-threaded native CPU version.

A hyperspectral band selection method based on sparse band attention network for maize seed variety identification

Effective soil spectral band selection and modeling methods can improve modeling accuracy. To establish a hyperspectral prediction model of soil organic matter (SOM) content, this study investigated a forested Eucalyptus plantation in Huangmian Forest Farm, Guangxi, China. The Ranger and Lasso algorithms were used to screen spectral bands. Subsequently, models were established using four algorithms: partial least squares regression, random forest (RF), a support vector machine, and an artificial neural network (ANN). The optimal model was then selected. The results showed that the modeling accuracy was higher when band selection was based on the Ranger algorithm than when it was based on the Lasso algorithm. ANN modeling had the best goodness of fit, and the model established by RF had the most stable modeling results. Based on the above results, a new method is proposed in this study for band selection in the early phase of soil hyperspectral modeling. The Ranger algorithm can be applied to screen the spectral bands, and ANN or RF can then be selected to construct the prediction model based on different datasets, which is applicable to establish the prediction model of SOM content in red soil plantations. This study provides a reference for the remote sensing of soil fertility in forests of different soil types and a theoretical basis for developing portable equipment for the hyperspectral measurement of SOM content in forest habitats.

Face recognition has attracted great interest due to its importance in many real-world applications. In this paper, we present a novel low-rank sparse representation-based classification (LRSRC) method for robust face recognition. Given a set of test samples, LRSRC seeks the lowest-rank and sparsest representation matrix over all training samples. Since low-rank model can reveal the subspace structures of data while sparsity helps to recognize the data class, the obtained test sample representations are both representative and discriminative. Using the representation vector of a test sample, LRSRC classifies the test sample into the class which generates minimal reconstruction error. Experimental results on several face image databases show the effectiveness and robustness of LRSRC in face image recognition.

Constrained energy minimization (CEM) is one of the most widely used target detection methods due to its simplicity and effectiveness. From a mathematical perspective, the detection result can be regarded as the summation of bands weighted by the corresponding components of the CEM operator. Based on this point of view, we propose a supervised band selection idea for CEM target detection. Three different band selection methods for CEM (namely, BSC1, BSC2 and BSC3) are presented and compared. Some comparative experiments based on real hyperspectral data are conducted to evaluate the performance of these methods (the presented and other two methods). The results show that BSC3 is the most effective band selection method. The CEM detection result is quite acceptable using only a few of the bands selected by BSC3. BSC1 and BSC2 are also more accurate than other compared band selection methods when the number of selected bands is relatively large (>18 in our experiments).

Face recognition has been widely studied due to its importance in different applications; however, most of the proposed methods fail when face images are occluded or captured under illumination and pose variations. Recently several low-rank dictionary learning methods have been proposed and achieved promising results for noisy observations. While these methods are mostly developed for single-modality scenarios, recent studies demonstrated the advantages of feature fusion from multiple inputs. We propose a multi-modal structured low-rank dictionary learning method for robust face recognition, using raw pixels of face images and their illumination invariant representation. The proposed method learns robust and discriminative representations from contaminated face images, even if there are few training samples with large intra-class variations. Extensive experiments on different datasets validate the superior performance and robustness of our method to severe illumination variations and occlusion.

The high dimensionality of hyperspectral imagery is a huge challenge for remote sensing data processing. Band selection utilizes the most distinctive and informative band subset to reduce data dimensions. Although band selection can significantly alleviate the computational burden, the process itself may be time consuming because it needs to take all pixels into consideration, especially when the image spatial size is larger. An improved band similarity-based band selection method is proposed for hyperspectral imagery target detection, which includes four steps: (1) bad bands are removed by data preprocessing; (2) several selected pixels are used for band selection instead of using all the pixels to reduce the computational complexity; (3) hyperspectral imagery is analyzed for target detection; and (4) the number of selected bands is determined by adjusting the threshold of similarity metric, to ensure target detection operators have the best performance with selected bands. In the example, the well-known adaptive coherence estimator detector was used to evaluate the effectiveness of the proposed band selection method. The receiver operating characteristics curves were plotted to prove the proposed algorithm quantitatively. The experimental results show that our method can yield a better result in target detection than other band selection methods.

Multiobjective band selection approach via an adaptive particle swarm optimizer for remote sensing hyperspectral images

The study attempts to investigate the significant spatiotemporal variations in the continental United States’ streamflow as a response to large-scale climate signals across multiple spectral bands (SBs). Using non-parametric (long-term) trend and (abrupt) shift detection tests, coupled with discrete wavelet transform, 237 unimpaired streamflow stations were analyzed over a study period of 62 years (1951 to 2012), looking at the water year and seasonal data, along with three discrete SBs of two, four, and eight years. Wavelet coherence analysis, derived from continuous wavelet transform, determined the association between the regional streamflow patterns and three large-scale climate signals, i.e., El Nino Southern Oscillation (ENSO), Pacific Decadal Oscillation (PDO), and Atlantic Multi-decadal Oscillation (AMO), across continuous SBs ranging from two to 16 years. The results indicated significant positive (negative) trends and shifts in the northeastern and north-central (northwestern) regions with an increase in the number of stations as the SBs increased. The spatiotemporal association between regional streamflow and the climate signals varied significantly (from no correlation, Rn2~ 0, to perfect correlation, Rn2~ 1.0) even amongst adjacent regions. Among the climate signals, ENSO showed the highest association (Rn2~ 1.0), having a consistent phase relationship with the regional streamflow patterns, especially in the higher SBs. PDO (with the least influence among the three signals) and AMO showed stronger associations, mostly in the lower SBs. These results may help explain the teleconnections between the climate signals and the US streamflow variations across multiple SBs, which may lead to improved regional flow regulations.