Deep learning enhanced quantum dot mosaic snapshot spectral imaging system.

Spectral imaging combines the advantages of spectroscopy as an analytical tool with the two-dimensional object visualization obtained by optical imaging. At present, spectral imaging systems suffer from severe scale-dependent geometric aberrations. This prevents the simultaneous realization of a wide field of view (FOV) and high resolution. Here we propose an ultra-wide-FOV high-resolution cascaded spectral imaging system (URIS), which captures an ultra-wide-FOV panchromatic image with a folding monocentric lens and obtains high-resolution spectral images after local aberration correction by the spectral imaging sub-channels. The key issue of how the position of the aperture stop affects the illumination uniformity of the sub-channels has been studied, contributing to achieving the spectral images stitching between adjacent sub-channels and improving the performance of the processed spectral image. The URIS is optimally designed with 19 spectral imaging sub-channels to cover a full FOV of about 110°. It achieves an excellent spatial resolution of 0.075 mrad and a high spectral resolution of 4 nm theoretically. An experiment is also demonstrated to verify the wide FOV and the uniformity of sub-channels of URIS, confirming the correctness of the theoretical study. URIS has great advantages in spectral imaging of large macroscopic scenes, inspiring various machine vision applications.

DMP-Net: Deep semantic prior compressed spectral reconstruction method towards intraoperative imaging of brain tissue.

Snapshot Compressive Spectral Imaging Systems (SCSI) compress the scenes by capturing 2D projections of the encoded underlying signals. A decoder, trained with pre-acquired datasets, reconstructs the spectral images. SCSI systems based on diffractive optical elements (DOE) provide a small form factor and the single DOE can be optimized in an end-to-end manner. Since the spectral image is highly compressed in a SCSI system based on a single DOE, the quality of image reconstruction can be insufficient for diverse spectral imaging applications. This work proposes a multishot spectral imaging system employing a double-phase encoding with a double DOE architecture (DoDo), to improve the spectral reconstruction performance. The first DOE is fixed and provides the benefits of the diffractive optical systems. The second DOE provides the variable encoding of the multishot architectures. The work presents a differentiable mathematical model for the multishot DoDo system and optimizes the parameters of the DoDo architecture in an end-to-end manner. The proposed system was tested using simulations and a hardware prototype. To obtain a low-cost system, the implementation uses a deformable mirror for the variable DOE. The proposed DoDo system shows an improvement of up to 4dB in PSNR in the reconstructed spectral images compared with the single DOE system.

Hyperspectral and multispectral imaging for evaluating food safety and quality

We combined a homebuilt multiphoton microscope and a prism-CCD based spectrograph to develop a spectral imaging system capable of imaging deep into live tissues. The spectral images originate from the two-photon autofluorescence of the tissue and second harmonic signal from the collagen fibers. A highly penetrating near-infrared light is used to excite the endogenous fluorophores via multiphoton excitation enabling us to produce high quality images deep into the tissue. We were able to produce 100-channel (330 nm to 600 nm) autofluorescence spectral images of live skin tissues in less than 2 minutes for each xy-section. The spectral images rendered in RGB (real) colors showed green hair shafts, blue cells, and purple collagen. Analysis on the optical signal degradation with increasing depth of the collagen second-harmonic signal showed 1) exponential decay behavior of the intensity and 2) linear broadening of the spectrum. This spectral imaging system is a promising tool for both in biological applications and biomedical applications such as optical biopsy.

If multiple objects are located closely, we often observe interreflection on the object surfaces. The interreflection is accompanied by a change in the appearance of the object surfaces. In this paper, we propose a method to estimate the spectral image components from the captured images of two closely apposed objects. A spectral imaging system is used for image capturing of both objects under uniform illumination. First, we describe the basis bispectral characteristics of a fluorescent object. Second, we model the interreflection of two objects with the bispectral characteristics. The surface appearance of each object was based on both reflected component and luminescent component. We show that the spectral composition of interreflection is determined by a series of multiplication of the Donaldson matrices of the two objects. The spectral components of one bounce interreflection can be limited to only four spectral functions. Third, an algorithm is developed to estimate the spectral image components from the observed spectral radiance images that are influenced by the interreflection. The algorithm is based on a two-step estimation procedure in which the spectral components are estimated separately in the outside and inside of the emission wavelength range. The feasibility of the proposed method is investigated using fluorescent samples in details.

In order to meet the needs of modern digital medical treatment for the quantification and precision of the diagnosis of burning wound, a spectral imaging microscope with long working distance which combined with hyperspectral imaging technology and microscopic imaging technology was designed. As a non-invasive diagnostic method, spectral detection has gained more in-depth applications in clinical medicine and biometric identification. On the basis of acquiring traditional two-dimensional planar images, the spectral microscopy imaging system further obtains the spectral information of the burned area, so as to perform quantitative analysis and diagnosis of the range, depth, and type of the wound. The system uses the principle of push broom to scan the burned site, and the Offner spectral imaging system is connected to the imaging microscope objective lens. It has a long working distance of 30mm, a spectral resolution of 5nm and good imaging quality. It has a compact, lightweight and miniaturized structure. It is of great significance for the application of spectral imaging technology in clinical medicine.

A spectral imager constructs a three-dimensional (two spatial and one spectral) image from a series of two-dimensional images. We discuss a technique for spectral imaging that multiplexes the spatial and spectral information on a staring focal plane and then demultiplexes the resulting imagery to obtain the spectral image. The spectral image consists of 100×100 spatial pixels and 25 spectral bands. The current implementation operates over the 3–5-μm band, but can easily be applied to other spectral regions. This approach to spectral imaging has high optical throughput and is robust to focal plane array nonuniformities. A hardware description, the mathematical development, and experimental results are presented.

A MLP architecture fusing RGB and CASSI for computational spectral imaging

Hyperspectral imaging (HSI) shows great promise for the detection and classification of several diseases, particularly in the fields of optical biopsy as applied to oncology, and functional retinal imaging in ophthalmology. In this paper, we discuss the application of HSI to the detection of retinal diseases and technological solutions that address some of the fundamental difficulties of spectral imaging within the eye. HSI of the retina offers a route to non-invasively deduce biochemical and metabolic processes within the retina. For example it shows promise for the mapping of retinal blood perfusion using spectral signatures of oxygenated and deoxygenated hemoglobin. Compared with other techniques using just a few spectral measurements, it offers improved classification in the presence of spectral cross-contamination by pigments and other components within the retina. There are potential applications for this imaging technique in the investigation and treatment of the eye complications of diabetes, and other diseases involving disturbances to the retinal, or optic-nerve-head circulation. It is well known that high-performance HSI requires high signal-to-noise ratios (SNR) whereas the application of any imaging technique within the eye must cope with the twin limitations of the small numerical aperture provided by the entrance pupil to the eye and the limit on the radiant power at the retina. We advocate the use of spectrally-multiplexed spectral imaging techniques (the traditional filter wheel is a traditional example). These approaches enable a flexible approach to spectral imaging, with wider spectral range, higher SNRs and lower light intensity at the retina than could be achieved using a Fourier-transform (FT) approach. We report the use of spectral imaging to provide calibrated spectral albedo images of healthy and diseased retinas and the use of this data for screening purposes. These images clearly demonstrate the ability to distinguish between oxygenated and deoxygenated hemoglobin using spectral imaging and this shows promise for the early detection of various retinopathies.

A single-pixel hyperspectral imager using two-stage Hadamard encoding

Spectral imaging has been extensively applied in many fields, including agriculture, environmental monitoring, biomedical diagnostics, etc. Thanks to the advances in sensor technology, spectral imaging systems nowadays provide finer and finer spectral resolution needed to characterize the spectral properties of materials. The high spectral resolution, however, raises an issue as the difference in spectral information between two adjacent wavelength bands is typically very small. As a result, much of the data in a scene seems to be redundant. However, critical information is embedded that often can be used to identify materials. This thesis aims at facilitating the analysis in spectral imaging by making use of pattern recognition techniques, on the one hand, to improve visualization, and on the other hand, to directly solve classification problems.

In this work, we propose a shared-modality multi-bandpass filtering (SMF) method for plug-and-play multispectral depth imaging (MDI). Unlike traditional multispectral filter array (MFA) cameras that require precise pixel-level alignment of spectral filters, our approach leverages a low-cost planar multi-bandpass (MBP) filter, enabling flexible and customizable multispectral imaging through simple real-time linear computation. A key advantage of our method is that, compared to conventional parallel systems, which rely on cross-modal registration to align spectral and depth images, robust shared-modal registration is achieved by integrating batch-fabricated MBPs across parallel cameras with a common passband, while also significantly reducing system volume. As a proof of concept, we developed a prototype system comprising two SMF cameras, capable of real-time 7-band spectral and depth imaging. The system achieves a full width at half maximum (FWHM) of 10-15 nm in spectral imaging, 1600 × 1300 spatial resolution, and a depth accuracy of 1.75 mm at 30 fps. Its capability was further demonstrated in an edge-computing autonomous driving scenario, validating the system's practical effectiveness.

LADMM-Net: An unrolled deep network for spectral image fusion from compressive data

Chapter 8 Spectral imaging and its use in the measurement of Förster resonance energy transfer in living cells