Dual-camera compressive hyperspectral imaging based on deep image prior and a guided filter.

Coded aperture snapshot spectral imager (CASSI) has been a popular spectral imaging architecture for its ability of capturing hyperspectral images with high temporal resolution. However, such snapshot imaging system entails a large sacrifice in the spatial resolution of the data cube, since only a small amount of light gets into the imager during one snapshot. Also, the spatial resolution of the CASSI system is limited by the pixel size (and amount) of the detector, while it is difficult to fabricate a dense detector with small pixel size, especially for infrared spectral bands. Super-resolution is an advanced post-processing technique to alleviate such problem by exploiting the prior information of the image. In this letter, we try to realize image super-resolution from the perspective of developing new form of measurements by taking advantage of a modified CASSI system equipped with a coded aperture with higher spatial resolution than the detector, merging the SR model into the hardware configuration. Then the original data cube can be reconstructed from lower resolution measurements, thus the super-resolution is realized during the compressive sensing reconstruction process. The new system can be achieved based on the classical CASSI architecture in two dual ways, one by replacing the coded aperture with a higher resolution one and the other by substituting the focal plane array (FPA) detector with a lower resolution one. The experiments show that, we can recover images of higher quality with the first modification of CASSI system above, simply using a higher resolution coded aperture.

Coded aperture snapshot spectral imaging (CASSI) has been demonstrated as a feasible solution to recover a 3D hyperspectral image by using a single 2D measurement. In this paper, we propose a new hybrid camera design for CASSI to capture high quality hyperspectral images while maintaining the snapshot advantage. Specifically, we employ a complementary RGB camera in conjunction with the CASSI system. The recorded RGB image can provide reliable spectral clue of the scene. By combining the coded hyperspectral information from the CASSI branch and the uncoded color information from the RGB branch, hyperspectral images can be reconstructed with high fidelity. Furthermore, by conducting demosaicing on the raw RGB image as a preprocessing procedure, even better performance can be achieved. Both theoretical analysis and simulation results show improved accuracy of the proposed method compared to the state-of-the-arts.

In the coded aperture snapshot spectral imaging (CASSI) system, the coded and compressed single-channel measurements need to be reconstructed into hyperspectral cubes. Existing discriminative models reconstruct the spectral cube by optimizing the mean squared error (MSE) between the ground truth and the predicted image, employing peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) as metrics to gauge the quality of reconstruction. However, these indicators often possess significant limitations in mimicking human visual perception and in discerning the impact of image distortions on perceived visual quality. In this article, a new model named CASSIDiff is proposed to reconstruct CASSI measurements, achieving advanced results in perceptual loss-based evaluation metrics such as learned perceptual image patch similarity (LPIPS) and Fréchet inception distance (FID). The diffusion model, which enjoys high accuracy and reliability in generative tasks, is used for the first time for the hyperspectral reconstruction task. A feature fusion mechanism based on discrete wavelet transform (DWT) is used to weaken the noise interference effect in the conditional diffusion model. Considering the interspectra similarity and long-range dependencies of hyperspectral data, the spatial-spectral attention mechanism is also introduced. Experiments show that CASSIDiff not only outperforms most existing algorithms in simulation datasets but also shows robustness to real data published and collected in our home-built CASSI system. The code and models are publicly available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/YifanSi/CASSIDiff</uri>.

Coded aperture snapshot spectral imager (CASSI) based on compressed sensing can capture a three-dimensional (3-D) data cube information in one snapshot without scanning, so it is a kind of novel spectral imaging technique with good prospects. In this paper, the data flow procession of two kind of CASSI were studied, which similarities and differences were depicted. The basic principle of the encoded representation of both spectral information and 2-D spatial information of a scene were presented. The numerical estimation techniques based on the compressive sensing theory were applied to reconstruct 3-D data cube.

Coded Aperture Snapshot Spectral Imaging (CASSI) multiplexes 3D Hyperspectral Images (HSIs) into a 2D sensor to capture dynamic spectral scenes, which, however, sacrifices the spatial information. Dual-Camera Compressive Hyperspectral Imaging (DCCHI) enhances CASSI by incorporating a Panchromatic (PAN) camera to compensate for the loss of spatial information in CASSI. However, the dual-camera structure of DCCHI disrupts the diagonal property of the product of the sensing matrix and its transpose, making it difficult to efficiently and accurately solve the data subproblem in closed-form and thereby hindering the application of model-based methods and Deep Unfolding Networks (DUNs) that rely on such a closed-form solution. To address this issue, we propose an Alternating Direction DUN, named ADRNN, which decouples the imaging model of DCCHI into a CASSI subproblem and a PAN subproblem. The ADRNN alternately solves data terms analytically and a joint prior term in these subproblems. Additionally, we propose a Cross Spectral Transformer (XST) to exploit the joint prior. The XST utilizes cross spectral attention to exploit the correlation between the compressed HSI and the PAN image, and incorporates Grouped-Query Attention (GQA) to alleviate the burden of parameters and computational cost brought by impartially treating the compressed HSI and the PAN image. Furthermore, we built a real DCCHI system and captured large-scale indoor and outdoor scenes for future academic research. Extensive experiments on both simulation and real datasets demonstrate that the proposed method achieves state-of-the-art (SOTA) performance. The code and datasets have been open-sourced at: https://github.com/ShawnDong98/ADRNN-XST.

Coded aperture snapshot spectral imaging (CASSI) faces the ill‐posed inverse problem, and reconstruction accuracy remains unsatisfactory despite various regularizers. Dual‐camera CASSI (DC‐CASSI) mitigates this problem by capturing panchromatic (PAN) images alongside compressed measurements. Although untrained deep neural networks (DNN) can leverage PAN imagery, they often struggle to preserve spectral integrity and ensure optimization stability. To address these dual issues, this paper proposes a panchromatic‐guided self‐supervised framework (PGSF). Its core is a novel spectral low‐rank guided spatial enhancement module (SLGSEM), which performs dual spatial‐semantic enhancement of network inputs by fusing PAN guidance with a learned low‐rank spectral prior. This module seamlessly integrates with various untrained DNN (joint network). Furthermore, the joint network is embedded as a regularizer within an ADMM‐based optimization algorithm, enabling self‐supervised parameter learning and progressive input refinement throughout iterations. Extensive experiments demonstrate that PGSF significantly enhances untrained DNN performance, achieving state‐of‐the‐art reconstruction in both quantitative metrics and visual quality without requiring any training data.

A novel compressive 3D imaging spectrometer based on the coded aperture snapshot spectral imager (CASSI) is proposed. By inserting a microlens array (MLA) into the CASSI system, one can capture spectral data of 3D objects in a single snapshot without requiring 3D scanning. The 3D spatio-spectral sensing phenomena is modelled by computational integral imaging in tandem with compressive coded aperture spectral imaging. A set of focal stack images is reconstructed from a single compressive measurement, and presented as images focused on different depth planes where the objects are located. The proposed optical system is demonstrated with simulations and experimental results.

Imaging spectroscopy involves the sensing of a large amount of spatial information across a multitude of wavelengths. Conventional approaches to hyperspectral sensing scan adjacent zones of the underlying spectral scene and merge the results to construct a spectral data cube. Push broom spectral imaging sensors, for instance, capture a spectral cube with one focal plane array (FPA) measurement per spatial line of the scene [1], [2]. Spectrometers based on optical bandpass filters sequentially scan the scene by tuning the bandpass filters in steps. The disadvantage of these techniques is that they require scanning a number of zones linearly in proportion to the desired spatial and spectral resolution. This article surveys compressive coded aperture spectral imagers, also known as coded aperture snapshot spectral imagers (CASSI) [1], [3], [4], which naturally embody the principles of compressive sensing (CS) [5], [6]. The remarkable advantage of CASSI is that the entire data cube is sensed with just a few FPA measurements and, in some cases, with as little as a single FPA shot.

Raman spectroscopy is an efficient method for detection of explosives even in small quantities. A laser can be combined with a Coded Aperture Snapshot Spectral Imaging (CASSI) system to collect Raman spectra from a surface at stand-off distances. The CASSI-system decrease the data collection time but instead increase the reconstruction time for the Raman image. Reconstruction of Raman spectra from an ensemble of compressed sensing measurements using standard reconstruction methods such as Total Variation (TV) is rather time consuming and limits the application domain for the technique. Novel machine learning approaches such as Deep Learning (DL) has lately been applied to reconstruction problems. We evaluate our earlier developed DL approach for reconstruction of Raman spectra from an ensemble of measurements formulated as a regression problem. The DL network is trained by minimizing a loss-function which is composed of two components: a reconstruction error and a re-projection error. The evaluated method is trained on simulated data where the training data has been generated using a transfer function. The transfer function has been developed to mimic the optical properties of a CASSI system. The DL network has been trained on different training sets with different levels of background noise, different number of materials in the scene and different spatial configurations of the materials in the scene. The reconstruction results using the DL network has been qualitatively evaluated on simulated data and the results are also compared to the Two-Step Iterative Shrinkage/Thresholding (TwIST) algorithm in terms of reconstruction quality and computation time. The reconstruction time for the DL are orders of magnitude lower than for TwIST without reducing the quality of the reconstructed Raman spectra.

Hyperspectral image classification (HIC) is an active research topic in remote sensing. Hyperspectral images typically generate large data cubes posing big challenges in data acquisition, storage, transmission and processing. To overcome these limitations, this paper develops a novel deep learning HIC approach based on compressive measurements of coded-aperture snapshot spectral imagers (CASSI), without reconstructing the complete hyperspectral data cube. A new kind of deep learning strategy, namely 3D coded convolutional neural network (3D-CCNN), is proposed to efficiently solve for the classification problem, where the hardware-based coded aperture is regarded as a pixel-wise connected network layer. An end-to-end training method is developed to jointly optimize the network parameters and the coded apertures with periodic structures. The accuracy of classification is effectively improved by exploiting the synergy between the deep learning network and coded apertures. The superiority of the proposed method is assessed over the state-of-the-art HIC methods on several hyperspectral datasets.

The hardware architecture of the coded aperture snapshot spectral imaging (CASSI) system is based on a coded mask design, resulting in a poor spatial resolution of the system. Therefore, we consider the use of a physical model of optical imaging and a jointly optimized mathematical model to design a self-supervised framework to solve the high-resolution-hyperspectral imaging problem. In this paper, we design a parallel joint optimization architecture based on a two-camera system. This framework combines the physical model of optical system and a joint optimization mathematical model, which takes full advantage of the spatial detail information provided by the color camera. The system has a strong online self-learning capability for high-resolution-hyperspectral image reconstruction, and gets rid of the dependence of supervised learning neural network methods on training data sets.

Earth observation (EO) data cubes have gained significance for their potential in providing comprehensive insights into dynamic earth phenomena. However, a significant challenge in their utilization is the interoperability among EO data cubes with varying dimensionalities. Existing efforts have primarily focused on enhancing analysis ready data cubes, yet cross-dimensional adaptivity remains relatively unexplored. This study proposes a novel approach to address this issue. The proposed method is based on a 3D adaptive data cube model constructed in a 3D adaptive space. The main objective is to achieve interoperability between EO data cubes of different dimensions while adhering to common geospatial standards. The core methodology involves cross-dimensional mapping, considering a 3D adaptive space containing spatial axes and adaptive axes. Importantly, the method applies to data cubes with the same coordinate reference system, resolution, and data type. To enable cross-dimensional mapping, a key requirement is the existence of a mapping function between the multidimensional space domain and the 3D adaptive space domain. When these conditions are met, data cubes with three or more dimensions can be interoperable. This becomes feasible by applying suitable serialization algorithms. This study demonstrates that data cubes can be successfully mapped into the proposed 3D adaptive data cube, achieving interoperability under specific conditions. This finding has significant implications in the field of EO data analysis, enabling seamless interaction between data cubes with different dimensions. The method not only facilitates cross-dimensional adaptability but also aligns with mainstream geospatial service standards. It is particularly suitable for multidimensional geospatial raster services based on the OGC WCS 2.0 standard. In conclusion, this study addresses a critical challenge in EO data analysis by proposing a 3D adaptive data cube model that promotes interoperability between data cubes of varying dimensions.

In this study, we propose a novel Deeply Optimized Adaptive chaotic synchronization algorithm system (DOA), which adopts the ideas of genetic algorithm, Deep Image Prior (DIP) network, Deep Convolutional Generative Adversarial Network (DCGAN) network, and slide mode control algorithm. Traditional Deep Learning (DL)-based methods perform well in complex multi-parameter operations, but training on large datasets is typically a complicated, time-consuming, and high-cost process. Such methods are also difficult to adapt to dynamic parameter changes. The algorithmic network model in the proposed DOA reduces reliance on large datasets by learning the deep mining methods of the data characteristics in DIP, and can adjust system parameters adaptively, accurately, and quickly, providing high synchronization efficiency and excellent stability over various chaotic signals. By applying Lyapunov stability theory, the robustness and global stability of the model in dynamic systems are proven. This paper also uses an advanced Recurrent Neural Network (RNN)-based chaotic synchronization system as a benchmark. The simulation results show that, when compared to the Recurrent Neural Network based synchronization system, the DOA architecture has significant advantages in robustness, convergence, and training over noisy channels. Experiments show that under strong noise (AWGN variance = 2) and parameter mismatch (±20 percent drift), the synchronization error of DOA (<0.3) is significantly lower than that of the RNN benchmark system (>1.5), and the training data volume is reduced by more than 30%. Simulation results show that, the DOA architecture has significant advantages in robustness, convergence, and training over noisy channels. The proposed DOA scheme improves the effect of chaotic synchronization and paves the way for the development of a new class of modulator schemes that meet the robustness, convergence, and training requirements for encrypted communication.

Enhanced Deep Image Prior (EDIP) is a lightweight framework designed to enhance hyperspectral images without the need for large-scale training datasets. Instead of relying on pretraining, it learns directly from each input image by leveraging the concept of Deep Image Prior (DIP), which allows the network structure itself to act as a prior for reconstruction and enhancement. The method employs a simplified U-Net architecture tailored for hyperspectral data, effectively capturing spatial–spectral correlations while maintaining computational efficiency. To further improve performance, EDIP incorporates scene-aware adaptation, enabling image-specific optimization that adjusts parameters dynamically to the characteristics of each scene. In addition, a basic yet effective spectral band fusion strategy is applied to preserve fine spectral and spatial details across different wavelengths, ensuring consistency in the enhanced output. The framework is complemented by an interactive web-based tool that visualizes the entire enhancement pipeline, providing intuitive before-and-after comparisons and enabling users to explore the effects of EDIP on diverse hyperspectral datasets.

The problem of efficiently compressing massive high-dimensional data cubes still waits for efficient solutions capable of overcoming well-recognized scalability limitations of state-of-the-art histogram-based techniques, which perform well on small-in-size low-dimensional data cubes, whereas their performance in both representing the input data domain and efficiently supporting approximate query answering against the generated compressed data structure decreases dramatically when data cubes grow in dimension number and size. To overcome this relevant research challenge, in this paper we propose LCS-Hist, an innovative multidimensional histogram devising a complex methodology that combines intelligent data modeling and processing techniques in order to tame the annoying problem of compressing massive high-dimensional data cubes. With respect to similar histogram-based proposals, our technique introduces (i) a surprising consumption of the storage space available to house the compressed representation of the input data cube, and (ii) a superior scalability on high-dimensional data cubes. Finally, several experimental results performed against various classes of data cubes confirm the advantages of LCS-Hist, even in comparison with those given by state-of-the-art similar techniques.