Polarimetric 3D integral imaging in photon-starved conditions

We propose a framework for three-dimensional (3D) object recognition and classification in very low illumination environments using convolutional neural networks (CNNs). 3D images are reconstructed using 3D integral imaging (InIm) with conventional visible spectrum image sensors. After imaging the low light scene using 3D InIm, the 3D reconstructed image has a higher signal-to-noise ratio than a single 2D image, which is a result of 3D InIm being optimal in the maximum likelihood sense for read-noise dominant images. Once 3D reconstruction has been performed, the 3D image is denoised and regions of interest are extracted to detect 3D objects in a scene. The extracted regions are then inputted into a CNN, which was trained under low illumination conditions using 3D InIm reconstructed images, to perform object recognition. To the best of our knowledge, this is the first report of utilizing 3D InIm and convolutional neural networks for 3D training and 3D object classification under very low illumination conditions.

Conventional sensors for hyperspectral imaging are limited by noise when operated in low illumination conditions. Recent advances in single-photon avalanche diode (SPAD) arrays provide image sensors capable of single photon counting imaging. Such devices provides camera systems which can operate under low illumination levels and high frame rates, with fully digital in-pixel processing capabilities. Here, we present a hyperspectral imaging system based on low level illumination of a target scene using nine LEDs, each with different emission wavelengths. The backscattered light is captured by a SPAD camera, which can recover spectral information based on the number of photon counts received for each illumination wavelength.

We present experiments to illustrate tracking of occluded objects using three-dimensional (3D) integral imaging (II). Tracking of heavily occluded objects by conventional two-dimensional (2D) image processing may be difficult owing to the superposition of occlusion noise and object details. The effects of occlusion are remedied by 3D computational II reconstruction. We use a summation of absolute difference (SAD) algorithm between pixels of consecutive frames of a moving object for 3D tracking. SAD algorithm is not robust for 2D images of occluded objects; 3D computation reconstruction of the scene allows implementation of a SAD algorithm by reducing occlusion effects. Experimental results demonstrate 3D tracking of occluded objects. To the best of our knowledge, this is the first report on 3D tracking of objects using II.

This paper presents a new method for extracting an elemental image array in three-dimensional (3D) integral imaging. To reconstruct 3D images in integral imaging, as the first step, a method is required to accurately extract an elemental image array from a raw captured image. Thus, several methods have been discussed to extract an elemental image array. However, the accuracy is sometimes degraded due to inaccurate edge detection, image distortions, optical misalignment, and so on. Especially, small pixel errors can deteriorate the performance of an integral imaging system with a lens array. To overcome the problem, we propose a postprocessing method for the accurate extraction of an elemental image array. Our method is a unified version of an existing method and proposed postprocessing techniques. The proposed postprocessing consists of re-calibration and lens array area detection. Our method reuses the results from an existing method, and it then improves the results via the proposed postprocessing techniques. To evaluate the proposed method, we perform optical experiments for 3D objects and provide the resulting images. The experimental results indicate that the proposed postprocessing techniques improve an existing method for extracting an elemental image array in integral imaging. Therefore, we expect the proposed techniques to be applied to various applications of integral imaging systems

Mixed content is a primary feature of current TV program making and consequently real-time computer generated video in autostereoscopic format is important if 3D TV is going to make an impact. In this paper, an approach for computer generating 3D integral imaging frames is proposed. The method is based on a 3D integral imaging ray-tracer containing 3D integral imaging parser, 3D camera model, and a 3D integral imaging renderer.

We overview a method of extracting 3D reconstructed polarimetric integral images from elemental images recorded in photon starved illumination conditions. Performance of the 3D integral imaging system significantly outperforms the conventional 2D imaging system.

In order to present the visual resolution effects of different integral imaging systems in three-dimensional (3D) display applications, a method of comparing the resolutions of integral imaging 3D display based on human vision is proposed. The relative resolution parameter is defined by analyzing the relation between the integral imaging 3D system and resolving power of human eyes on the best viewing condition, and its relationship with the visual resolution effect viewed in the integral imaging 3D display system is analyzed. The results indicate that the size of lens array has an important influence on the visual resolution effect in the integral imaging 3D display system. According to the theoretical results of two examples show that the difference in relative resolution parameter between the systems is 1.75 times of that of traditional one. The experimental results are consistent with the theoretical analyses. The method proposed in this paper could have a certain directive significance for 3D display resolution evaluation.

CMOS photon detectors (CPDs) are recently proposed photon sensing devices utilizing the latest CMOS image sensor (CIS) technologies [1]. CPDs are non-electron-multiplying devices, whose pixels have a fully depleted photo diode and have a readout noise of sub electron RMS even at room temperature. Using a 15μm pixel CPD test device coupled to a CsI(Tl) scintillator, we successfully obtained photon-count X-ray images. A Hamamatsu Photonics scintillator with 150μm CsI(Tl) layer coupled to fiber optic plate (FOP) of 3mm thickness is diced in dry condition and directly glued to the sensor surface. X-ray photons are injected from an X-ray tube with accelerating voltage of 30kV and 45kV using W target. Each X-ray photon creates a scintillation light spot in the captured images, where the injected position and photon energy are determined by integrating multiple pixel outputs at that spot. X-ray energy distributions were obtained at both 30kV and 45kV with reasonable differences. Modulated transfer function (MTF) of over 0.7 at 10LP/mm was achieved by mapping injected positions at 30kV. Photon-count images for slanted-edge MTF measurements as well as 10LP/mm of X-ray test chart were achieved. Those photon-count images were compared with conventional energy integrating images obtained with the same sensing device. Both image types confirmed superior resolution with photon-counting. This indirect X-ray photon counting technique using CPD has a potential of getting critical X-ray information for medical applications by achieving accurate injected positions of X-ray photons and their energies simultaneously.

The quality of the integral 3D images created by a 3D imaging system was improved by combining multiple LCDs to utilize a greater number of pixels than that possible with one LCD. A prototype of the display device was constructed by using four HD LCDs. An integral photography (IP) image displayed by the prototype is four times larger than that reconstructed by a single display. The pixel pitch of the HD display used is 55.5 μm, and the number of elemental lenses is 212 horizontally and 119 vertically. The 3D image pixel count is 25,228, and the viewing angle is 28^°. Since this method is extensible, it is possible to display an integral 3D image of higher quality by increasing the number of LCDs. Using this integral 3D display structure makes it possible to make the whole device thinner than a projector-based display system. It is therefore expected to be applied to the home television in the future.

The photon counting image collected under 10− 4 lux environment has a degraded image quality due to background noises and other problems. Bayesian estimation is a classical approach for photon counting image restoration and regularization has also been widely used in image processing. However, the regularization method is not suitable for photon counting images with extremely lack of information, and the recovery effect of Bayesian estimation in images mixed with unknown noise is not ideal. The main contribution of this paper is that on the basis of Bayesian estimation, the regularization method is introduced to solve the problem of restoring photon counting images mixed with unknown noise under 10− 4 lux environment. The original part is that the gamma distribution of the expected value of photon counting is used as its prior condition, and the error function is expressed as the form of the norm to establish the objective function. Through an approximate iterative solution, the optimal estimation of the photon counting expectation is carried out to achieve the optimal restoration of the photon counting image. Experiments demonstrate that the background noise is effectively removed and the image quality is improved after restoring photon counting images. Also, the final result of the proposed method is superior to other comparative methods in multiple evaluation indexes and achieved better effects.

We overview temporally encoded single shot polarimetric integral imaging with nonlinear correlation for optical detection in turbid water. The overviewed system enhances the detection capabilities compared to conventional 2D and 3D integral imaging methods.

The degree of linear polarization (DOLP) expressions at any polarizer direction (PD) was deduced based on the Stokes vector and Mueller matrix. The average gray level range of the target units (20x20 pixels material) under triple-waveband and low illumination condition was obtained with 46 kinds of experimental materials. This paper mainly explored that the influence factors of the error and stability of multi-PD DOLP was researched by the numerical analysis, and outdoors experiments were carried out to demonstrate the analysis method. Studies have shown that when detection angle was near specular reflection direction, when interval angle of PD between angle A and angle B, between angle A and angle C were set as the range (40°≤ΦAB ≤80°, ΦAC =2.06ΦAB), the standard deviation of DOLP was lower, but also polarization information obtained under low illumination condition was stable, and the numerical analysis was consistent with the experimental results.

Novel diffraction limit implied geometric lens imaging model based on a finite number of pixels for integral imaging display

<div class="section abstract"><div class="htmlview paragraph">Advanced Driver Assistance Systems (ADAS) have become an inevitable part of most of the modern cars. Their use is mandated by regulations in some cases; and in other cases where vehicle owners have become more safety conscious. Vision / camera based ADAS systems are widely in use today. However, it is to be noted that the performance of these systems is depends on the quality of the image/video captured by the camera. Low illumination is one of the most important factors which degrades image quality. In order to improve the system performance under low illumination, it is required to first enhance the input images/frames. In this paper, we propose an image enhancement algorithm that would automatically enhance images to a near ideal condition. This is accomplished by mapping features taken from images acquired under ideal illumination conditions on to the target low illumination images/frames.</div><div class="htmlview paragraph">The proposed method consists of four steps a) Pre- processing b) a coarse level segmentation of the input image, c) searching for an appropriate images from the database and d) adaptive color transfer. Since our algorithm performs appropriate adaptive modification to various regions, the quality of resultant image is good even under low illumination condition. We have also done a quantitative evaluation using the entropy and contrast based measures. Results prove that our method performs much better both quantitatively and qualitatively when compared to the standard state of the art image enhancement tools that are widely used.</div></div>

In recent years, e‐readers have spread to many countries. Most of e‐readers have a reflective type of display called e‐paper. The general view is that under conditions of low illuminance, readability of the e‐paper display is poor. However, most recent e‐readers have a built‐in light system. This system would improve the readability of the e‐paper display under low illuminance conditions. We carried out experiments to evaluate the readability e‐paper devices using different systems. In the experiments, we conducted subjective evaluations under staged illuminance conditions by using English words. This study found that e‐paper devices with built‐in light under low illuminance condition were superior in terms of readability.