Multi-Resolution Compressive Spectral Imaging Reconstruction from Single Pixel Measurements.

Compressive spectral imaging (CSI) allows the acquisition of the spectral information of a three-dimensional scenes by using coded projections in a sensor with lower dimension. However, the compressed sampling of information with simultaneously high spatial and high spectral resolution demands expensive high-resolution sensors. One of the main challenges in CSI is to obtain a high-quality image of high-resolution reconstructions using low-cost architectures. Single pixel camera is an approach that has had a high impact in spectroscopy, due to its low-cost implementation compared to CSI architectures with 2D sensors. On the other hand, recent works have been shown that image fusion using measurements from a CSI sensor based on side information leads to improvement in the quality of the fused image. This work proposes a spectral image fusion methodology for increasing the spatio-spectral resolution through side information and at the same time improve the reconstruction quality of the data cube with a low-cost architecture, optimizing the similarity of the reconstructed spectral image with each sensor. Simulations and experimental results for the proposed methodology show that improve the quality of the reconstruction in up to 11 dB with respect to the traditional approach of upsampling the single pixel image reconstruction through bilinear interpolation.

Compressive spectral imaging (CSI) enables the acquisition of spectral and spatial information of a scene using fewer projected measurements than traditional scanning approaches. Recently, research efforts have focused on obtaining high-resolution spectral images via expensive detectors and sophisticated CSI devices. Alternatively, high-resolution spectral images can be obtained using side information or fusion of compressed measurements, without significantly increasing acquisition costs. Indeed, these approaches retrieve improved resolution images applying iterative and computationally expensive algorithms. This paper proposes the fusion of compressed measurements obtained from two state-of-the-art CSI systems, the single-pixel camera (SPC) and the three-dimensional coded aperture snapshot imaging system (3D-CASSI), such that high-resolution images can be obtained by exploiting detailed spectra provided by the SPC and high spatial resolution of the 3D-CASSI. Specifically, a noniterative reconstruction algorithm is proposed, based on the fact that the spatial-spectral data lie in a low-dimensional subspace. In contrast to related works, the proposed approach relies on implementable CSI systems. Simulations and experimental results show the effectiveness of the proposed method compared to similar approaches, both in reconstruction quality and complexity. Specifically, the proposed method is up to 5.6 times faster than its counterparts and provides comparable quality of attained reconstructions.

In recent years there has been an increasing interest in compressive imaging devices that capture spectral images with high spatial and spectral resolution with as few as a single snapshot. Nonetheless, there exists an intrinsic trade-off between spatial and spectral resolution which degrades one at the expense of the other. To alleviate this, state-of-art systems have relied on multiple snapshots attained via dynamically programmable spatial light modulators (SLM), or on side information from a second sensor attained via beam-splitters or stereo setups, which suffer from occlusion and demand registration tasks. This work proposes a compressive snapshot spectral imaging architecture that builds on the ideas of side information, but relies on a single image sensor. Specifically, we propose to split the incoming light into two paths, such as in a conventional interferometer, with one path dispersing the coded light, while the other just relaying it, so as to multiplex a traditional coded-and-dispersed measurement with a modulated grayscale version of the 3D cube in the image sensor. We have coined the latter, virtual side information. The proposed optical system is versatile in the sense that it can move from a fully-encoding system, to a spatially modulated grayscale camera, by switching/blocking the splitting response of the beam-splitters. Furthermore, we propose a computational algorithm to recover the underlying spectral cube exploiting the high spatial resolution provided by the virtual side information. We demonstrate through simulations from over fifty spectral images, and via an experimental proof-of-concept implementation, that the proposed imaging system together with the computational algorithm represents an efficient alternative to acquire spectral images without relying on additional sensors.

Compressive spectral imaging has garnered significant attention for its ability to effectively enhance the captured spatial and spectral information. Predominant methods, based on compressive sensing, typically formulate the imaging task as a constrained optimization problem and rely on hand-crafted priors to model the sparsity of spectral images. However, these approaches often suffer from suboptimal performance due to the inherent difficulty of identifying an appropriate transform space where spectral images exhibit sparsity. To overcome this limitation, we propose a novel convolutional sparse coding-inspired untrained network prior for fast and adaptive identification of the sparse transform domain and compressible signal. Specifically, a Lightweight Convolutional Thresholding sparse Coding (LCTC) network is designed as the sparse transform domain, with its inputs interpreted as sparse coefficients. Crucially, both the transform domain and its coefficients are solved in a self-supervised learning manner. Furthermore, we demonstrate that LCTC prior can be seamlessly incorporated into the iterative optimization algorithm as a Plug-and-Play (PnP) regularization. Both the LCTC and PnP-LCTC exhibit superior performance compared to previous methods. Experiments under various scenarios validate the effectiveness and efficiency of our approach.

Compressive spectral imaging (CSI) acquires coded projections of a spectral image by performing a modulation of the data cube followed by a spectral-wise integration. To avoid the spectral image reconstruction procedure, this paper proposes a classification approach that extracts features directly from multi-sensor CSI measurements. Particularly, the proposed method obtains the features by considering the spectral information extracted from Hyperspectral CSI measurements, and the local spatial information extracted by clustering the Multispectral CSI measurements using a superpixel algorithm. This approach is evaluated on Pavia University and Salinas Valley datasets. Extensive simulations show that considering the local spatial information boosts the overall accuracy up to 3% in comparison with traditional approaches that only uses the spectral information. Furthermore, the computation time of the approach that reconstructs, fuses and classifies takes approximately 87.43 [s], while classifying directly from multi-sensor compressive measurements takes only 0.74 [s], achieving similar classification results.

LADMM-Net: An unrolled deep network for spectral image fusion from compressive data

Compressive spectral imaging (CSI) acquires compressed observations of a spectral scene by applying different coding patterns at each spatial location and then performing a spectral-wise integration. Relying on compressive sensing, spectral image reconstruction is achieved by using nonlinear and relatively expensive optimization-based algorithms. In the CSI literature, several works have focused on improving reconstructions quality by properly designing the set of coding patterns. However, signal recovery is not actually necessary in many signal processing applications. For instance, assuming that compressed measurements with similar characteristics lie on the same subspace, unsupervised methods such as subspace clustering can be used to separate them into the same cluster. Since the structure of compressed measurements is defined by the applied codification, it is possible to improve clustering performance. This paper proposes to design a set of coding patterns such that inter-class and intra-class data structure is preserved after the CSI acquisition in order to improve clustering results directly on the compressed domain. To validate the coding pattern design, an algorithm based on sparse subspace clustering (SSC) is proposed to perform clustering on the compressed measurements. The proposed algorithm adds a three-dimensional (3-D) spatial regularizer to the SSC problem exploiting the spatial correlation of spectral images. In general, an overall accuracy up to 83.81% is obtained, when noisy measurements are assumed. In addition, a difference of at most 4% in terms of overall accuracy was observed when comparing the clustering results obtained by the full 3-D data with those achieved using CSI measurements acquired with the proposed coding pattern design.

DMP-Net: Deep semantic prior compressed spectral reconstruction method towards intraoperative imaging of brain tissue.

A spectral imager constructs a three-dimensional (two spatial and one spectral) image from a series of two-dimensional images. We discuss a technique for spectral imaging that multiplexes the spatial and spectral information on a staring focal plane and then demultiplexes the resulting imagery to obtain the spectral image. The spectral image consists of 100×100 spatial pixels and 25 spectral bands. The current implementation operates over the 3–5-μm band, but can easily be applied to other spectral regions. This approach to spectral imaging has high optical throughput and is robust to focal plane array nonuniformities. A hardware description, the mathematical development, and experimental results are presented.

Spectral classification allows material labeling based on spectral information. Single-pixel cameras (SPCs) have been used as a low-cost solution for acquiring spectral images, providing high-resolution spectral and low-resolution spatial information. Also, diffractive optical cameras (DOCs) based on multilevel phase masks (MPMs) can acquire spectral features to perform classification tasks. Traditional spectral classification approaches have not incorporated SPCs and DOCs into a single optical architecture. This work proposes a dual optical system based on SPC and DOC for spectral classification. Specifically, the height map in the MPM and the deep neural network parameters are jointly learned from end-to-end (E2E) optimization. The proposed method contains an optical layer that describes the dual system, a fusion layer that estimates the spectral image, and a classification network that labels the materials over spectral datasets. The simulation results show an improvement of up to 3% in classification metrics compared to other optical architectures.

High-Fidelity compressive spectral image reconstruction through a novel Non-Convex Non-Local Low-Rank tensor approximation model

Coded aperture snapshot spectral imager (CASSI), efficiently captures 3D spectral images by sensing 2D projections of the scene. While CASSI offers a substantial reduction in acquisition time, compared to traditional scanning optical systems, it requires a reconstruction post-processing step. Furthermore, to obtain high-quality reconstructions, an accurate propagation model is required. Notably, CASSI exhibits a variant spatio-spectral sensor response, making it difficult to acquire an accurate propagation model. To address these inherent limitations, this work proposes to learn a deep non-linear fully differentiable propagation model that can be used as a regularizer within an optimization-based reconstruction algorithm. The proposed approach trains the non-linear spatially-variant propagation model using paired compressed measurements and spectral images, by employing side information only in the calibration step. From the deep propagation model incorporation into a plug-and-play alternating direction method of multipliers framework, our proposed method outperforms traditional CASSI linear-based models. Extensive simulations and a testbed implementation validate the efficacy of the proposed methodology.

A novel compressive 3D imaging spectrometer based on the coded aperture snapshot spectral imager (CASSI) is proposed. By inserting a microlens array (MLA) into the CASSI system, one can capture spectral data of 3D objects in a single snapshot without requiring 3D scanning. The 3D spatio-spectral sensing phenomena is modelled by computational integral imaging in tandem with compressive coded aperture spectral imaging. A set of focal stack images is reconstructed from a single compressive measurement, and presented as images focused on different depth planes where the objects are located. The proposed optical system is demonstrated with simulations and experimental results.

Spectral imaging has been extensively applied in many fields, including agriculture, environmental monitoring, biomedical diagnostics, etc. Thanks to the advances in sensor technology, spectral imaging systems nowadays provide finer and finer spectral resolution needed to characterize the spectral properties of materials. The high spectral resolution, however, raises an issue as the difference in spectral information between two adjacent wavelength bands is typically very small. As a result, much of the data in a scene seems to be redundant. However, critical information is embedded that often can be used to identify materials. This thesis aims at facilitating the analysis in spectral imaging by making use of pattern recognition techniques, on the one hand, to improve visualization, and on the other hand, to directly solve classification problems.

In compressive spectral imaging, three-dimensional spatio-spectral data cubes are recovered from two-dimensional projections. The quality of the compressive-sensing-based reconstruction is dependent on the coherence of the sensing matrix, which is determined by the system projection and the sparse prior. Studies on the optimization of the system projection, which mainly deals with the coded aperture, successfully decreases the coherence of the sensing matrix and improves the reconstruction quality. However, the optimization of the sparse prior considering the relationship between the system projection and the sparse prior remains a challenge. In this paper, we propose a gradient-descent-based sparse prior optimization algorithm for the coherence minimization of the sensing matrix in compressive spectral imaging. The Frobenius norm coherence is introduced as the cost function for the optimization, and the overcomplete dictionary is chosen as the sparse prior to solve the optimal sparse representation in the reconstruction as it provides higher degree of freedom for optimization compared to common orthogonal bases. The optimized dictionary effectively decreases the coherence of the sensing matrix from 0.880 to 0.604 and significantly improves the quantitative image quality metrics of the reconstructed hyperspectral images with the corresponding peak signal-to-noise ratio (PSNR) increased by 9 dB, the structural similarity (SSIM) above 0.98, and the spectrum angular mapper (SAM) below 0.1. Furthermore, the requirement of the sampling snapshots is reduced, which is shown by similar image quality metrics between the reconstructed hyperspectral images of only 1 snapshot with the optimized dictionary and of more than 5 snapshots with the non-optimized dictionary.