Exploiting intervoxel covariance as prior information in millimeter-wave computational imaging systems

In the modern tactical imaging environment, new computational imaging (CI) systems and algorithms are being used to improve speed and accuracy for detection tasks. Therefore, a measurement technique is needed to predict the performance of complex non-shift invariant EO/IR imaging systems, including CI systems. Detection performance of traditional imaging systems can be modeled using current system metrics and measurements such as Modulation Transfer Function (MTF), Signal to Noise (SNR), and instantaneous Field of View (iFOV). In this correspondence, we propose a technique to experimentally measure a detection sensitivity metric for non-traditional CI systems. The detection sensitivity metric predicts the upper bound of linear algorithm performance though evaluation of a matched filter. The experimental results are compared with theoretical expected values though the Night Vision Integrated Performance Model (NV-IPM). Additionally, we demonstrate the experimental results for a variety of imaging systems (IR, visible, and color), target sizes and orientations, as well as SNR values. Our results demonstrate how this detection sensitivity metric can be measured to provide additional insight into the final system performance.

Imaging systems using aspheric imaging lenses with complementary computation can deliver performance unobtainable in conventional imaging systems. These new imaging systems, termed Wavefront coded imaging systems, use specialized optics to capture a coded image of the scene. Decoding the intermediate image provides the "human-usable" image expected of an imaging system. Computation for the decoding step can be made completely transparent to the user with today's technology. Real-time Wavefront coded systems are feasible and cost-effective. This "computational imaging" technology can be adapted to solve a wide range of imaging problems. Solutions include the ability to provide focus-free imaging, to increase the field of view, to increase the depth of read, to correct for aberrations (even in single lens systems), and to account for assembly and temperature induced misalignment. Wavefront coded imaging has been demonstrated across a wide range of applications, including microscopy, miniature cameras, machine vision systems, infrared imaging systems and telescopes.

Computational imaging is a proven strategy for obtaining high-quality images with fast acquisition rates and simpler hardware. Metasurfaces provide exquisite control over electromagnetic fields, enabling the radiated field to be molded into unique patterns. The fusion of these two concepts can bring about revolutionary advances in the design of imaging systems for security screening. In the context of computational imaging, each field pattern serves as a single measurement of a scene; imaging a scene can then be interpreted as estimating the reflectivity distribution of a target from a set of measurements. As with any computational imaging system, the key challenge is to arrive at a minimal set of measurements from which a diffraction-limited image can be resolved. Here, we show that the information content of a frequency-diverse metasurface aperture can be maximized by design, and used to construct a complete millimeter-wave imaging system spanning a 2 m by 2 m area, consisting of 96 metasurfaces, capable of producing diffraction-limited images of human-scale targets. The metasurfacebased frequency-diverse system presented in this work represents an inexpensive, but tremendously flexible alternative to traditional hardware paradigms, offering the possibility of low-cost, real-time, and ubiquitous screening platforms.

Metasurface has drawn extensive attention due to its capability of modulating light with a high degree of freedom through ultrathin and sub-wavelength optical elements, and metalens, as one of its important applications, promises to replace the bulky refractive optics, facilitating the imaging system light-weight and compact characteristics. Besides, computer-generated holography (CGH) is of substantial interest for three-dimensional (3D) imaging technology by virtue of its ability of restoring the whole optical wave field and re-constructing the true 3D scene. Consequently, the combination of metalens and CGH holds transformative potential in enabling the miniaturization of 3D imaging systems. However, its imaging performance is subject to the aberrations and speckle noises originating from the metalens and CGH. Inspired by recent progress that computational imaging can be applied to close the gap, a novel full-color imaging system, adopting end-to-end joint optimization of metalens and CGH for high imaging quality, is proposed in this paper. The U-net based network as the pre-processing adjusts weights to make the holographic reconstruction offset imaging defects, incorporating the imaging processing into the step of generating hologram. Optimized by deep learning, the proposed imaging system is capable of full-color imaging with high fidelity in a compact form factor, envisioned to take an essential step towards the high-performance miniaturized imaging system.

Computational imaging modalities support a simplification of the active architectures required in an imaging system and these approaches have been validated across the electromagnetic spectrum. Recent implementations have utilized pseudo-orthogonal radiation patterns to illuminate an object of interest-notably, frequency-diverse metasurfaces have been exploited as fast and low-cost alternative to conventional coherent imaging systems. However, accurately measuring the complex-valued signals in the frequency domain can be burdensome, particularly for sub-centimeter wavelengths. Here, computational imaging is studied under the relaxed constraint of intensity-only measurements. A novel 3D imaging system is conceived based on 'phaseless' and compressed measurements, with benefits from recent advances in the field of phase retrieval. In this paper, the methodology associated with this novel principle is described, studied, and experimentally demonstrated in the microwave range. A comparison of the estimated images from both complex valued and phaseless measurements are presented, verifying the fidelity of phaseless computational imaging.

International audience

Abstract. As remote sensing and variable-rate technology are becoming more available for aerial applicators, practical methodologies for effective integration of these technologies are needed for site-specific aerial applications of crop production and protection materials. The objectives of this study were to demonstrate how to integrate an airborne multispectral imaging system and an IntelliStar variable-rate aerial application system for site-specific management of the winter weed henbit. The airborne imaging system was used to acquire natural color and near-infrared (NIR) images of a fallow field near College Station, Texas, for mapping the infestation of henbit shortly before herbicide application. The images were then rectified, classified, and converted to a binary prescription map with the full application rate on infested areas and the zero rate on non-infested areas. The variable-rate aerial application system mounted on an agricultural aircraft was used to apply glyphosate over the field based on the prescription map. Airborne imagery was collected 14 days after the herbicide application. The as-applied map along with the prescription map and the post-application imagery were used to assess the performance of the site-specific application. Spatial and statistical analysis results showed that the imaging system was effective for mapping henbit infestations and for assessing the performance of site-specific herbicide application, and that the variable-rate system accurately delivered the product at the desired rate to the prescribed areas for effective control of the weed. The methodology and results from this study will be useful for aerial applicators to incorporate airborne imaging and variable-rate application systems into their aerial application business to increase their capabilities and profits. Keywords: Aerial application, As-applied map, Henbit, Imaging system, Prescription map, Variable-rate technology.

Breakthroughs brought about by two‐dimensional (2D) materials in the field of photodetection have opened up new possibilities in infrared imaging. However, challenges still exist in fabricating high‐density detector arrays using such materials, which are essential for traditional imaging systems. In this study, we present a state‐of‐the‐art computing imaging system that utilizes a MoTe2/Si self‐powered photodetector coupled with flexible Hadamard modulation algorithms. This system demonstrates remarkable capability to produce high‐quality images in the shortwave infrared (SWIR) band, surpassing the capabilities of devices based on alternative material systems. The exceptional infrared imaging capability primarily stems from the MoTe2/Si photodetector's inherent features, including an ultra‐wide spectral range (265–1550 nm) and extremely high sensitivity (linear dynamic range (LDR) up to 123 dB, responsivity (R) up to 0.33 A W–1, external quantum efficiency (EQE) up to 43% and a specific detectivity (D*) exceeding 2.9 × 1011 Jones). Moreover, the imaging system demonstrates the ability to achieve high‐quality edge imaging of objects in the SWIR band (1550 nm), even in strong scattering environments and under low sampling rate conditions (sampling rate of 25%). We believe that this work will effectively advance the application scope of 2D materials in the field of computational imaging in SWIR bands.image

Origami is the art of paper folding that allows a single flat piece of paper to assume different 3D shapes depending on the fold patterns and the sequence of folding. Using the principles of origami along with computation imaging technique the authors demonstrate a versatile shape‐morphing microwave imaging array with reconfigurable field‐of‐view and scene‐adaptive imaging capability. Microwave/millimeter‐wave based array imaging systems are expected to be the workhorse for sensory perception of future autonomous intelligent systems. The imaging capability of a planar array‐based systems operating in complex scattering conditions have limited field‐of‐view and lack the ability to adaptively reconfigure resolution. To overcome this, here, deviations from planarity and isometry are allowed, and a shape‐morphing computational imaging system is demonstrated. Implemented on a reconfigurable Waterbomb origami surface with 22 active metasurface panels that radiate near‐orthogonal modes across 17–27 GHz, capability to image complex 3D objects in full details minimizing the effects of specular reflections in diffraction‐limited sparse imaging with scene adaptability, reconfigurable cross‐range resolution, and field‐of‐view is demonstrated. Such electromagnetic origami surfaces, through simultaneous surface shape‐morphing ability (potentially with shape‐shifting electronic materials) and electromagnetic field programmability, opens up new avenues for intelligent and robust sensing and imaging systems for a wide range of applications.

The novel concept of filter-lens array (FLA) is introduced in this paper. Measured results for a Ka-band design show that a typical FLA can meet system requirements for a wide range of applications. FLAs present a compact and simple means of focusing and filtering at millimeter-wave frequencies, and can replace the standard combinations of dielectric lenses and bandpass filters. They are expected to find application in power combining, medium-angle beam-steering, and millimeter-wave (mm-wave) imaging systems.

Medical images are acquired for the purpose of gathering information regarding the disease state of a patient. The quality of the image should be a measure of how a specified task may be performed given the resulting image. Image quality is task dependent; one imaging system may provide images that allow better task performance than another system's for one task, but worse for another task. Determining the quality of an imaging system involves specifying the task of interest, the range of objects that will be imaged, and the observer (human or machine reader) who will perform the task. In radiography it has become common to use Fourier measures to describe image quality through a combination of Fourier‐based measures of system resolution and noise. The Line Spread Function, when Fourier transformed, yields the Modulation Transfer Function (MTF) as a measure of system resolution. Use of the MTF to describe a system's resolution is appropriate when the imaging system is shift‐invariant, that is, the line spread function is independent of position. Image noise is described in the Fourier domain by the Noise Power Spectrum (NPS). We can use the NPS to describe the noise in an imaging system whenever the noise properties are stationary, meaning that the variance and correlations between fluctuations in different locations are not dependent on their absolute location. These Fourier measures of system resolution and noise are commonly combined to give an overall figure of merit known as the Detective Quantum Efficiency (DQE). The DQE describes the signal‐to‐noise transfer characteristics of an imaging system as long as those assumptions of shift‐invariance and noise stationarity are valid. In digital systems the image of an object is affected by the location of the object with respect to the pixel grid. In other words, the detector pixels cause the system to be shift‐variant. In this case the MTF does not describe the resolution properties completely. This is particularly the case for tomographic systems, where the point‐response function is quite position‐dependent. Moreover, in real imaging systems the noise properties will depend on position, most notably because real patients always have structure. So, the Fourier‐domain noise concept of Noise Power Spectrum does not capture all the properties of noise for real imaging systems and tasks. In this talk we will consider a general treatment of the imaging system as a mapping of information from the patient to the data, without the assumptions inherent in Fourier analysis. We will discuss approaches to the characterization of system resolution and noise in terms of position‐dependent parameters. Once such position‐dependent imaging properties of a system are determined, we can use them to calculate measures of image quality that summarize the usefulness of the images for the performance of various visual tasks.

A simple multispectral classification method for the identification of systemically diseased chickens was developed and tested between two different imaging systems. An image processing algorithm was developed to define and locate the region of interest (ROI) as classification areas on the image. The average intensity was calculated for each classification area of the chicken image. A decision tree algorithm was used to determine threshold values for each classification areas. The wavelength of 540 nm was used for image differentiation purpose. There were 164 wholesome and 176 systemically diseased chicken images collected using the first imaging system, and 332 wholesome and 318 systemically diseased chicken images taken by the second imaging system. The differentiation thresholds, generated by the decision tree method, based on the images from the first imaging system were applied to the images from the second imaging system, and vice versa. The accuracy from evaluation was 95.7% for wholesome and 97.7% of systemically diseased chickens for the first image batch, and 99.7% for wholesome and 93.5% for systemically diseased chickens for the second image batch. The result showed that using single wavelength and threshold, this simple classification method can be used in automated on-line applications for chicken inspection.

Conventional wisdom in model-based computational imaging incorporates physics-based imaging models, noise characteristics, and image priors into a unified Bayesian framework. Rapid advances in deep learning have inspired a new generation of data-driven computational imaging systems with performances even better than those of their model-based counterparts. However, the design of learning-based algorithms for computational imaging often lacks transparency, making it difficult to optimize the entire imaging system in a complete manner.

The optical transfer function (OTF), which correspond to the frequency domain, is of fundamental importance in characterizing imaging ability in optical imaging systems. By using the Collins formula and the optical transfer matrix theory, a simple and unified analytical expression of OTF is presented in any ghost imaging (GI) system. By comparing the spectrum distribution of OTF between different GI systems, one can predict which imaging system can provide better imaging quality when the same imaging object is used. Theoretical and experimental results are also implemented to verify our results.

A millimeter-wave (mmW) 3-D imaging system using high scanning-rate leaky-wave antennas in a frequency-diverse multiple-input–multiple-output (MIMO) antenna configuration is proposed. A novelty of the approach is the use of high scanning-rate leaky-wave antennas combined with MIMO to obtain both frequency and spatial diversities. Furthermore, an extended Rytov approximation (xRA), recently shown to provide accurate reconstructions for high permittivity and electrically large-sized low-loss objects, is also included in the approach. Compared with frequency-diversity-only and spatial-diversity-only imaging systems, the proposed system can provide enhanced imaging performance by leveraging both spatial and frequency diversities simultaneously. In addition, the imaging time and hardware cost can be significantly reduced, resulting in a low-cost mmW imaging system. The formulation for the 3-D imaging system is simplified to a scalar form by leveraging the polarization of the leaky-wave antennas and the polarization dominance of the scattered wave components, allowing xRA to be formulated. Sensing capacity and correlation coefficient evaluation methods are used to estimate the performance of the high scanning-rate leaky-wave MIMO imaging system. The scalar formulation also allows us to straightforwardly compare MIMO microwave imaging with MIMO radar to highlight the commonality between them. Numerical and experimental examples demonstrate that the proposed system with xRA can accurately estimate the contrast function amplitude and the positions of dielectric scatterers in a 3-D imaging region using two transmit and 11 receive leaky-wave antennas and 49 frequency samples within the 37.2–42-GHz frequency range.