High-dynamic range compressive spectral imaging by adaptive filtering

Coded aperture compressive imaging array applied for surveillance systems

We have developed a compressive hyperspectral imaging system that is based on single-pixel camera architecture. We have incorporated the developed system in a scanning white-light interferometer (SWLI) and showed that replacing SWLI’s CCD-based camera by the compressive hyperspectral imaging system, we have access to high-resolution multispectral images of interferometer’s fringes. Using these multi-spectral images, the system is capable of simultaneous spectroscopy of the surface, which can be used, for example, to eliminate the effect of surface contamination and providing new spectral information for fringe signal analysis which could be used to reduce the need for vertical scan, therefore making height measurement more tolerant to object’s position.

Compressive sensing enables efficient acquisition of hyperspectral images (HSIs) by assuming high redundancy on natural scenes. Several reconstruction algorithms have been proposed to retrieve the underlying image, and most of them take advantage of the spatial and spectral correlations. However, reconstruction may not be necessary in certain applications such as land cover classification. Instead of knowing the full image, researchers are interested in features that could be extracted directly from the compressed measurements, which provide high inference capabilities. Low-rank (LR) matrix approximation has been widely used in feature extraction (FE), because it reduces the data dimension and computational cost. Therefore, in this paper, compressive hyperspectral imaging and FE are combined in a framework for HSI classification using an LR matrix approximation model. In the proposed framework, the compressed measurements are acquired from a single-pixel spectrometer. Instead of using the traditional high-complexity reconstruction model, an LR matrix factorization problem is formulated. The LR problem maximizes the posterior distribution with respect to the feature space and coefficients, and it is numerically solved based on an alternating optimization strategy. By incorporating spatial information, the numerical procedure minimizes the total variational of the feature coefficients subject to an orthogonality constraint for the feature space. Experiments on real HSIs show that the proposed approach can provide equally competitive classification results when compared to the traditional approach that performs FE and classification on the recovered images.

Compressive radar imaging has attracted considerable attention because it substantially reduces imaging time through directly compressive sampling. However, a problem that must be addressed for compressive radar imaging systems is the high computational complexity of reconstruction of sparse signals. In this paper, a novel algorithm, called two-dimensional (2D) normalized iterative hard thresholding (NIHT) or 2D-NIHT algorithm, is proposed to directly reconstruct radar images in the matrix domain. The reconstruction performance of 2D-NIHT algorithm was validated by an experiment on recovering a synthetic 2D sparse signal, and the superiority of the 2D-NIHT algorithm to the NIHT algorithm was demonstrated by a comprehensive comparison of its reconstruction performance. Moreover, to be used in compressive radar imaging systems, a 2D sampling model was also proposed to compress the range and azimuth data simultaneously. The practical application of the 2D-NIHT algorithm in radar systems was validated by recovering two radar scenes with noise at different signal-to-noise ratios, and the results showed that the 2D-NIHT algorithm could reconstruct radar scenes with a high probability of exact recovery in the matrix domain. In addition, the reconstruction performance of the 2D-NIHT algorithm was compared with four existing efficient reconstruction algorithms using the two radar scenes, and the results illustrated that, compared to the other algorithms, the 2D-NIHT algorithm could dramatically reduce the computational complexity in signal reconstruction and successfully reconstruct 2D sparse images with a high probability of exact recovery.

Coded aperture snapshot spectral imager (CASSI) has been a popular spectral imaging architecture for its ability of capturing hyperspectral images with high temporal resolution. However, such snapshot imaging system entails a large sacrifice in the spatial resolution of the data cube, since only a small amount of light gets into the imager during one snapshot. Also, the spatial resolution of the CASSI system is limited by the pixel size (and amount) of the detector, while it is difficult to fabricate a dense detector with small pixel size, especially for infrared spectral bands. Super-resolution is an advanced post-processing technique to alleviate such problem by exploiting the prior information of the image. In this letter, we try to realize image super-resolution from the perspective of developing new form of measurements by taking advantage of a modified CASSI system equipped with a coded aperture with higher spatial resolution than the detector, merging the SR model into the hardware configuration. Then the original data cube can be reconstructed from lower resolution measurements, thus the super-resolution is realized during the compressive sensing reconstruction process. The new system can be achieved based on the classical CASSI architecture in two dual ways, one by replacing the coded aperture with a higher resolution one and the other by substituting the focal plane array (FPA) detector with a lower resolution one. The experiments show that, we can recover images of higher quality with the first modification of CASSI system above, simply using a higher resolution coded aperture.

The desire for larger and larger format arrays for astronomical observatories -- both ground and space based -- has fueled the development of detector, readout, and hybrid Focal Plane Array (FPA) technology that has paved the way for later development of tactical and strategic arrays for military applications. Since 1994, Raytheon has produced megapixel readouts and FPAs for Infrared Astronomy. In 1999 Raytheon demonstrated a revolutionary approach to photolithography called Reticle Image Composition Lithography (RICL) that opened the door to very large format FPAs in state of the art sub-micron CMOS processes. The first readout processed using the patented RICL technique was a 4.2 megapixel readout for astronomy. We present the design and performance of several 4.2 megapixel (2048 x 2048) readout arrays for visible and infrared astronomy applications. The first of these arrays are fabricated in a workhorse 2 μm CMOS process that is optimized for low temperature operation (down to as low as 6 Kelvin). Most recently Raytheon has developed a scaleable 2,048 x 2,048 high density array for several ground based astronomical applications. This array can be manufactured in any m x n multiple of a basic 1024 (V) x 512 (H) pixel array core. The primary design is a 2 x 4 array to yield a 2,048 x 2,048 format array. This same design can be extended to at least a 4,096 x 4,096 format array -- an incredible 16.7 megapixel array! These readouts are compatible with a wide range of detector types including InSb, HgCdTe, and Si detectors. The use of hybrid technology -- even for the visible wavebands -- allows 100% optical fill factors to be achieved. The design and performance of these megapixel class detectors, readouts, and FPAs will be presented.

Compressive hyperspectral imaging is based on the fact that hyperspectral data is highly redundant. However, there is no symmetry between the compressibility of the spatial and spectral domains, and that should be taken into account for optimal compressive hyperspectral imaging system design. Here we present a study of the influence of the ratio between the compression in the spatial and spectral domains on the performance of a 3D separable compressive hyperspectral imaging method we recently developed.

Noise equivalent temperature difference performance of an IR detector in a hybrid focal plane array

Development of focal plane arrays (FPA) for mm wavelength and THz radiation is presented in this paper. The FPA is based upon inexpensive neon indicator lamp Glow Discharge Detectors (GDDs) that serve as pixels in the FPA. It was shown in previous investigations that inexpensive neon indicator lamps GDDs are quite sensitive to mm wavelength and THz radiation. The diameter of GDD lamps are typically 3-6 mm and thus the FPA can be diffraction limited. Development of an FPA using such devices as detectors is advantageous since the costs of such a lamp is around 30-50 cents per lamp, and it is a room temperature detector sufficiently fast for video frame rates. Recently a new 8×8 GDD FPA VLSI board was designed, constructed, and experimentally tested. First THz images as well as DSP methods using this GDD FPA are demonstrated. Super resolution was achieved by moving the 8×8 pixel board appropriately in the image plane so that 32X32 pixel images are also obtained and shown here, with much improved image quality because of much reduced pixelization distortion.

There are some difficulties in the development of uncooled focal plane array (FPA) detectors due to the absence of full simulation model which reflects the characterization of FPA detectors by variations of various parameters. In this paper we propose the simulator for the both readout integrated circuit (ROIC) and bolometer FPA which is based on a thermal equivalence equation of bolometer and mathematical modeling of optical and electrical part in infrared sensor system. The simulator shows the characteristics and the behaviors of individual components of infrared sensor system in the transient-state and steady-state. We present here the simulation results for output characteristics of detectors owing to variations of parameters induced non-uniformity in FPA detectors and find the dominant parameter to be the leading source non-uniformity in FPA detectors. We also present the simulation results for some typical ROICs to cancel the bias-heating which wastes most of the dynamic range of infrared sensor system. These show the effectiveness of compensation for the bias-heating according to variations of parameters. Using the proposed simulator we can expect the quantitative amount of non-uniformity due to the statistical variations in various processing steps and design of ROIC components. It can be used for the systematic design of infrared sensor system which cannot be performed in fabrication procedure.

This paper will discuss and compare recent refractive and catodioptric imager designs developed and manufactured at SunSpace for Multi Sensor Satellite Imagers with Panchromatic, Multi-spectral, Area and Hyperspectral sensors on a single Focal Plane Array (FPA). These satellite optical systems were designed with applications to monitor food supplies, crop yield and disaster monitoring in mind. The aim of these imagers is to achieve medium to high resolution (2.5m to 15m) spatial sampling, wide swaths (up to 45km) and noise equivalent reflectance (NER) values of less than 0.5%. State-of-the-art FPA designs are discussed and address the choice of detectors to achieve these performances. Special attention is given to thermal robustness and compactness, the use of folding prisms to place multiple detectors in a large FPA and a specially developed process to customize the spectral selection with the need to minimize mass, power and cost. A refractive imager with up to 6 spectral bands (6.25m GSD) and a catodioptric imager with panchromatic (2.7m GSD), multi-spectral (6 bands, 4.6m GSD), hyperspectral (400nm to 2.35μm, 200 bands, 15m GSD) sensors on the same FPA will be discussed. Both of these imagers are also equipped with real time video view finding capabilities. The electronic units could be subdivided into the Front-End Electronics and Control Electronics with analogue and digital signal processing. A dedicated Analogue Front-End is used for Correlated Double Sampling (CDS), black level correction, variable gain and up to 12-bit digitizing and high speed LVDS data link to a mass memory unit.

Recently there have been developed a host of approaches for more efficient hyperspectral imaging acquisition through hardware and algorithmic innovations. Unfortunately, these are all requiring the use of large, very-sensitive focal plane array detection and as such become prohibitively expensive. Infrared imaging similarly suffers from costly focal plane arrays to achieve high resolution images and therefore have not been employed for commercial machine vision applications. This proposal is a multifold approach to examine and overcome these obstacles and realize efficient means of both compressive infrared and hyperspectral computer recognition. Specifically, this project will address new compressive optical designs for infrared and spatio-spectral imaging and create the accompanying new deep learning algorithms for high-speed object detection/recognition directly from the compressive measurements. Our approach of a hardware-software hybrid will provide an increasingly detailed and systematic means for visually contextualizing objects and their surrounding environments in previously accessible but very costly portions of the electromagnetic spectrum.

Compressive spectral imaging (CSI) acquires compressed observations of a spectral scene by applying different coding patterns at each spatial location and then performing a spectral-wise integration. Relying on compressive sensing, spectral image reconstruction is achieved by using nonlinear and relatively expensive optimization-based algorithms. In the CSI literature, several works have focused on improving reconstructions quality by properly designing the set of coding patterns. However, signal recovery is not actually necessary in many signal processing applications. For instance, assuming that compressed measurements with similar characteristics lie on the same subspace, unsupervised methods such as subspace clustering can be used to separate them into the same cluster. Since the structure of compressed measurements is defined by the applied codification, it is possible to improve clustering performance. This paper proposes to design a set of coding patterns such that inter-class and intra-class data structure is preserved after the CSI acquisition in order to improve clustering results directly on the compressed domain. To validate the coding pattern design, an algorithm based on sparse subspace clustering (SSC) is proposed to perform clustering on the compressed measurements. The proposed algorithm adds a three-dimensional (3-D) spatial regularizer to the SSC problem exploiting the spatial correlation of spectral images. In general, an overall accuracy up to 83.81% is obtained, when noisy measurements are assumed. In addition, a difference of at most 4% in terms of overall accuracy was observed when comparing the clustering results obtained by the full 3-D data with those achieved using CSI measurements acquired with the proposed coding pattern design.

Block-wise motion detection using compressive imaging system

Compressed sensing or compressive sampling (CS) is a new framework for simultaneous data sampling and compression which was proposed by Candes, Donoho, and Tao several years ago. Ever since the advent of a single-pixel camera, one of the CS applications - compressive imaging (CI, also referred as feature-specific imaging) has aroused more interest of numerous researchers. However, it is still a challenging problem to choose a simple and efficient measurement matrix in such a hardware system, especially for large scale image. In this paper, we propose a new measurement matrix whose rows are the odd rows of N order Hadamard matrix and discuss the validity of the matrix theoretically. The advantage of the matrix is its universality and easy implementation in the optical domain owing to its integer-valued elements. In addition, we demonstrate the validity of the matrix through the reconstruction of natural images using Orthogonal Matching Pursuit (OMP) algorithm. Due to the limitation of the memory of the hardware system and personal computer which is used to simulate the process, it is impossible to create such a large matrix that is used to conduct large scale images. In order to solve the problem, the block-wise notion is introduced to conduct large scale images and the experiments results present the validity of this method.