Unmixing hyperspectral skin data using non-negative matrix factorization

The hyperspectral and multispectral image fusion process, commonly known as the multisharpening process (also known as the hypersharpening process), results in the creation of unobservable high spectral and spatial resolution hyperspectral remote sensing data, by combining (or fusing) a hyperspectral image, characterized by a high spectral resolution, with, especially, a high spatial resolution multispectral one. There are several approaches in the literature performing such a process, in particular, those based on the spectral unmixing concept. The latter fusion methods aim to extract pure-material spectra from the high spectral resolution hyperspectral image and combine them with their corresponding proportions extracted from the high spatial resolution multispectral image. However, most of these methods, which mainly consider the standard linear mixing model, do not address the spectral variability phenomenon, precisely due to the considered data mixing model, which does not handle variations in illumination and atmospheric conditions or material composition inducing spectral variability. Neglecting such a phenomenon may lead to errors in the fusion process, thus decreasing the spatial and spectral qualities of sharpened products. In this paper, a new hyperspectral and multispectral data fusion method, based on the linear spectral unmixing concept that deals with spectral variability, is proposed. This approach, based on a multiplicative Nonnegative Matrix Factorization (NMF) technique, substantially extends the multiplicative Coupled NMF (CNMF) algorithm by considering a linear data mixing model that addresses the considered phenomenon. Indeed, the introduced approach minimizes new cost functions involving matrices with specific structures in order to deal with the spectral variability phenomenon. The proposed method is applied to synthetic and real data, and its effectiveness is evaluated with established performance criteria. Experimental results show that the designed algorithm globally yields sharpened hyperspectral data with fine spectral and spatial fidelities. Also, these results demonstrate that the proposed technique globally outperforms the tested literature approaches.

Hyperspectral images (HSI) have high spectral and low spatial resolutions. However, multispectral images (MSI) usually have low spectral and high spatial resolutions. In various applications HSI with high spectral and spatial resolutions are required. In this paper, a new method for spatial resolution enhancement of HSI using high resolution MSI based on sparse coding and linear spectral unmixing (SCLSU) is introduced. In the proposed method (SCLSU), high spectral resolution features of HSI and high spatial resolution features of MSI are fused. In this case, the sparse representation of some high resolution MSI and linear spectral unmixing (LSU) model of HSI and MSI is simultaneously used in order to construct high resolution HSI (HRHSI). The fusion process of HSI and MSI is formulated as an ill-posed inverse problem. It is solved by the Split Augmented Lagrangian Shrinkage Algorithm (SALSA) and an orthogonal matching pursuit (OMP) algorithm. Finally, the proposed algorithm is applied to the Hyperion and ALI datasets. Compared with the other state-of-the-art algorithms such as Coupled Nonnegative Matrix Factorization (CNMF) and local spectral unmixing, the SCLSU has significantly increased the spatial resolution and in addition the spectral content of HSI is well maintained.

To evaluate the effect of varying spectral resolution on image quality of high spectral and spatial resolution (HiSS) images. Eight women with suspicious breast lesions and six healthy volunteers were scanned using echo-planar spectroscopic imaging (EPSI) at 1.5 Tesla with 0.75- to 1-mm in-plane resolution and 2.3- to 2.6-Hz spectral resolution. Time domain data were truncated to obtain proton spectra in each voxel with varying (2.6-83.3 Hz) resolution. Images with intensity proportional to water signal peak heights were synthesized. Changes in water signal line shape following contrast injection were analyzed. Fat suppression is optimized at approximately 10-Hz spectral resolution and is significantly improved by removal of wings of the fat resonance. This was accomplished by subtracting a Lorentzian fit to the fat resonance from the proton spectrum. The water resonance is often inhomogeneously broadened, and very high spectral resolution is necessary to resolve individual components. High spectral resolution is required for optimal contrast in anatomic features with very high T(2)* (e.g., within a lesion) and for detection of often subtle effects of contrast agents on water signal line shape. Despite a trade-off between the spectral resolution and signal-to-noise ratio, it is beneficial to acquire data at the highest spectral resolution currently attainable at 1.5 Tesla.

Hyperspectral imaging has become a significant source of valuable data for astronomers over the past decades. Current instrumental and observing time constraints allow direct acquisition of multispectral images, with high spatial but low spectral resolution, and hyperspectral images, with low spatial but high spectral resolution. To enhance scientific interpretation of the data, we propose a data fusion method which combines the benefits of each image to recover a high spatio-spectral resolution datacube. The proposed inverse problem accounts for the specificities of astronomical instruments, such as spectrally variant blurs. We provide a fast implementation by solving the problem in the frequency domain and in a low-dimensional subspace to efficiently handle the convolution operators as well as the high dimensionality of the data. We conduct experiments on a realistic synthetic dataset of simulated observation of the upcoming James Webb Space Telescope, and we show that our fusion algorithm outperforms state-of-the-art methods commonly used in remote sensing for Earth observation.

Feature extraction is an important research topic in hyper-spectral image (HSI) classification. However, most of feature extraction methods only extract low-level features, which makes them not perform well in the applications of HSI. In this paper, we have proposed a non-negative matrix factorization (NMF) based deep feature extraction algorithm, namely deep NMF. Deep NMF tries to construct a deep feature representation by cascading multiple NMFs. Reconstruction residual of NMF is passed layer by layer to reduce information loss. Meanwhile, passing residuals between layers can construct a feature hierarchy from coarse to fine. Furthermore, activation functions are applied between adjacent layers to enhance the ability of non-linear feature extraction. Experimental results have also shown that our algorithm is computationally efficient and effective for HSI classification.

Abstract. Hyperspectral image enhancement has been a concern for the remote sensing society for detailed end member detection. Hyperspectral remote sensor collects images in hundreds of narrow, continuous spectral channels, whereas multispectral remote sensor collects images in relatively broader wavelength bands. However, the spatial resolution of the hyperspectral sensor image is comparatively lower than that of the multispectral. As a result, spectral signatures from different end members originate within a pixel, known as mixed pixels. This paper presents an approach for obtaining an image which has the spatial resolution of the multispectral image and spectral resolution of the hyperspectral image, by fusion of hyperspectral and multispectral image. The proposed methodology also addresses the band remapping problem, which arises due to different regions of spectral coverage by multispectral and hyperspectral images. Therefore we apply algorithms to restore the spatial information of the hyperspectral image by fusing hyperspectral bands with only those bands which come under each multispectral band range. The proposed methodology is applied over Henry Island, of the Sunderban eco-geographic province. The data is collected by the Hyperion hyperspectral sensor and LISS IV multispectral sensor.

For many remote sensing applications it is preferable to have images with both high spectral and spatial resolutions. On this regards, hyperspectral and multispectral images have complementary characteristics in terms of spectral and spatial resolutions. In this paper we propose an approach for the fusion of low spatial resolution hyperspectral images with high spatial resolution multispectral images in order to obtain superresolution (spatial and spectral) hyperspectral images. The proposed approach is based on the assumption that, since both hyperspectral and multispectral images acquired on the same scene, the corresponding endmembers should be the same. On a first step the hyperspectral image is spectrally down-sampled in order to match the multispectral one. Then an endmember extraction algorithm is performed on the down-sampled hyperspectral image and the successive abundance estimation is performed on the multispectral one. Finally, the extracted endmembers are up-sampled back to the original hyperspectral space and then used to reconstruct the superresolution hyperspectral image according to the abundances obtained from the multispectral image.

Hyperspectral vision beyond 3D: A review

Most modern satellites can provide two types of images: 1) panchromatic (PAN) image and 2) multispectral (MS) image. The former has high spatial resolution and low spectral resolution, while the latter has high spectral resolution and low spatial resolution. To obtain images with both high spectral and spatial resolution, pansharpening has emerged to fuse the spatial information of the PAN image and the spectral information of the MS image. However, most pansharpening methods fail to preserve spatial and spectral information simultaneously. In this article, we propose a framelet-based convolutional neural network (CNN) for pansharpening which makes it possible to pursue both high spectral and high spatial resolution. Our network consists of three subnetworks: 1) feature embedding net; 2) feature fusion net; and 3) framelet prediction net. Different from conventional CNN methods directly inferring high-resolution MS images, our approach learns to predict their framelet coefficients from available PAN and MS images. The introduction of multilevel feature aggregation and hybrid residual connection makes full use of spatial information of PAN image and spectral information of MS image. Quantitative and qualitative experiments at reduced- and full-resolution demonstrate that the proposed method achieves more appealing results than other state-of-the-art pansharpening methods. The source code and trained models are available at https://github.com/TingMAC/FrMLNet.

Hyperspectral unmixing is an important image interpre-tation technique that aims to estimate the pure constituent materials (endmembers) and their corresponding fractional abundances in each mixed pixel. Nonnegative matrix factorization (NMF) has attracted a lot of attention because of its ability to solve mixed pixel scenarios. The sparse NMF method achieves better unmixing results thanks to its full use of the sparse characteristics of the data. However, most existing sparse NMF unmixing techniques lack the consid-eration of spatial information. In fact, hyperspectral images contain intrinsic geometric information as well as rich spatial information. In this paper, a spatial graph regularized nonneg-ative matrix factorization unmixing framework (SGNMF) is established. For the proposed SGNMF, on the one hand, the graph regularization is introduced to characterize the latent manifold structure of the data, and on the other hand, the spatial weighting factor is used to mine the spatial correlation between pixels. The optimization problem of the SGNMF model can be solved by a multiplicative iterative rule. Exper-imental results on synthetic data sets indicate that the newly proposed SGNMF method is able to produce better results than other advanced spectral unmixing algorithms.

Hyperspectral unmixing is an important technique for analyzing remote sensing images which aims to obtain a collection of endmembers and their corresponding abundances. In recent years, non-negative matrix factorization (NMF) has received extensive attention due to its good adaptability for mixed data with different degrees. The majority of existing NMF-based unmixing methods are developed by incorporating additional constraints into the standard NMF based on the spectral and spatial information of hyperspectral images. However, they neglect to exploit the nature of imbalanced pixels included in the data, which may cause the pixels mixed with imbalanced endmembers to be ignored, and thus the imbalanced endmembers generally cannot be accurately estimated due to the statistical property of NMF. To exploit the information of imbalanced samples in hyperspectral data during the unmixing procedure, in this paper, a cluster-wise weighted NMF (CW-NMF) method for the unmixing of hyperspectral images with imbalanced data is proposed. Specifically, based on the result of clustering conducted on the hyperspectral image, we construct a weight matrix and introduce it into the model of standard NMF. The proposed weight matrix can provide an appropriate weight value to the reconstruction error between each original pixel and the reconstructed pixel in the unmixing procedure. In this way, the adverse effect of imbalanced samples on the statistical accuracy of NMF is expected to be reduced by assigning larger weight values to the pixels concerning imbalanced endmembers and giving smaller weight values to the pixels mixed by majority endmembers. Besides, we extend the proposed CW-NMF by introducing the sparsity constraints of abundance and graph-based regularization, respectively. The experimental results on both synthetic and real hyperspectral data have been reported, and the effectiveness of our proposed methods has been demonstrated by comparing them with several state-of-the-art methods.

Hyperspectral images (HSIs) usually have high spectral and low spatial resolution. Conversely, multispectral images (MSIs) usually have low spectral and high spatial resolution. The fusion of HSI and MSI aims to create spectral images with high spectral and spatial resolution. In this paper, we propose a fusion algorithm by combining linear spectral unmixing with the local low-rank property. By taking advantage of the local low-rank property, we first partition the corresponding spectral image into patches. For each patch pair, we cast the fusion problem as a coupled spectral unmixing problem that extracts the abundance and the endmembers of MSI and HSI, respectively. It then updates the abundance and the endmember through an alternating update algorithm. In fact, the convergence of the alternative update algorithm can be mathematically and empirically supported. We also propose a multiscale postprocessing procedure to combine fusion results obtained under different patch sizes. In experiments on three data sets, the proposed fusion algorithms outperformed state-of-the-art fusion algorithms in both spatial and spectral domains.

Hyperspectral Imaging (HSI) combines microscopy and spectroscopy to assess the spatial distribution of spectroscopically active compounds in objects, and has diverse applications in food quality control, pharmaceutical processes, and waste sorting. However, due to the large size of HSI datasets, it can be challenging to analyze and store them within a reasonable digital infrastructure, especially in waste sorting where speed and data storage resources are limited. Additionally, as with most spectroscopic data, there is significant redundancy, making pixel and variable selection crucial for retaining chemical information. Recent high-tech developments in chemometrics enable automated and evidence-based data reduction, which can substantially enhance the speed and performance of Non-Negative Matrix Factorization (NMF), a widely used algorithm for chemical resolution of HSI data. By recovering the pure contribution maps and spectral profiles of distributed compounds, NMF can provide evidence-based sorting decisions for efficient waste management. To improve the quality and efficiency of data analysis on hyperspectral imaging (HSI) data, we apply a convex-hull method to select essential pixels and wavelengths and remove uninformative and redundant information. This process minimizes computational strain and effectively eliminates highly mixed pixels. By reducing data redundancy, data investigation and analysis become more straightforward, as demonstrated in both simulated and real HSI data for plastic sorting.

Recent advances and new guidelines on hyperspectral and multispectral image fusion

Hyperspectral images due to their higher spectral resolution are increasingly being used for various remote sensing applications including information extraction at subpixel level. Typically whenever an object gets spectrally resolved but not spatially, mixed pixels in the images result. Numerous man made and/or natural disparatetar gets may thus occur inside such mixed pixels giving rise to subpixel target detection problem. Various spectral unmixing models such as linear mixture modeling (LMM) are in vogue to recover components of a mixed pixel. Spectral unmixing outputs both the endmember spectrum and their corresponding a bundance fractions inside the pixel. It, however, does not provide spatial distribution of these abundance fractions within a pixel. This limits the applicability of hyperspectral data for subpixel target detection. In this paper, a new inverse Euclidean distance based super-resolution mapping method has been presented. In this method, the subpixel target detection is performed by adjusting spatial distribution of abundance fraction within a pixel of an hyperspectral image. Results obtainedat different resolutions indicate that super-resolution mapping may effectively be utilized in enhancing the target detection at sub-pixel level. Defence Science Journal, 2013, 63(1), pp.63 -68 , DOI:http://dx.doi.org/10.14429/dsj. 63.3765