Computer Generated Content for 3D TV

We propose a framework for three-dimensional (3D) object recognition and classification in very low illumination environments using convolutional neural networks (CNNs). 3D images are reconstructed using 3D integral imaging (InIm) with conventional visible spectrum image sensors. After imaging the low light scene using 3D InIm, the 3D reconstructed image has a higher signal-to-noise ratio than a single 2D image, which is a result of 3D InIm being optimal in the maximum likelihood sense for read-noise dominant images. Once 3D reconstruction has been performed, the 3D image is denoised and regions of interest are extracted to detect 3D objects in a scene. The extracted regions are then inputted into a CNN, which was trained under low illumination conditions using 3D InIm reconstructed images, to perform object recognition. To the best of our knowledge, this is the first report of utilizing 3D InIm and convolutional neural networks for 3D training and 3D object classification under very low illumination conditions.

The quality of the integral 3D images created by a 3D imaging system was improved by combining multiple LCDs to utilize a greater number of pixels than that possible with one LCD. A prototype of the display device was constructed by using four HD LCDs. An integral photography (IP) image displayed by the prototype is four times larger than that reconstructed by a single display. The pixel pitch of the HD display used is 55.5 μm, and the number of elemental lenses is 212 horizontally and 119 vertically. The 3D image pixel count is 25,228, and the viewing angle is 28^°. Since this method is extensible, it is possible to display an integral 3D image of higher quality by increasing the number of LCDs. Using this integral 3D display structure makes it possible to make the whole device thinner than a projector-based display system. It is therefore expected to be applied to the home television in the future.

We develop a method for obtaining 3D polarimetric integral images from elemental images recorded in low light illumination conditions. Since photon-counting images are very sparse, calculation of the Stokes parameters and the degree of polarization should be handled carefully. In our approach, polarimetric 3D integral images are generated using the Maximum Likelihood Estimation and subsequently reconstructed by means of a Total Variation Denoising filter. In this way, polarimetric results are comparable to those obtained in conventional illumination conditions. We also show that polarimetric information retrieved from photon starved images can be used in 3D object recognition problems. To the best of our knowledge, this is the first report on 3D polarimetric photon counting integral imaging.

This paper describes a fast multi-scale vessel enhancement filter in 3D medical images. For efficient review of the vascular information, clinicians need rendering the 3D vascular information as a 2D image. Generally, the maximum intensity projection (MIP) is a useful and widely used technique for producing a 2D image from the 3D vascular data. However, the MIP algorithm reduces the conspicuousness for small and faint vessels owing to the overlap of non-vascular structures. To overcome this invisibility, researchers have examined the multi-scale vessel enhancement filter based on a combination of the eigenvalues of the 3D Hessian matrix. This multi-scale vessel enhancement filter produces higher contrast. However, it is time-consuming and requires high cost computation due to large volume of data and complex 3D convolution. For fast vessel enhancement, we propose a novel multi-scale vessel enhancement filter using 3D integral images and 3D approximated Gaussian kernel. This approximated kernel looks like cube but it is not exact cube. Each layer of kernel is approximated 2D Gaussian second order derivative by dividing it into three rectangular regions whose sum is integer. 3D approximated kernel is a pile of these 2D box kernels which are normalized by Frobenius norm. Its size fits to vessel width in order to achieve better visualization of the small vessel. Proposed method is approximately five times faster and produces comparable results with previous multi-scale vessel enhancement filter.

Novel diffraction limit implied geometric lens imaging model based on a finite number of pixels for integral imaging display

There is a phenomenon that a 3D image appears in proportion to a focus distance when something is watched through a convex lens. An adjustable focus lens which can control the focus distance of the convex lens is contrived and applied to 3D TV. We can watch 3D TV without eyeglasses. The 3D TV image meets the NTSC standard. A parallax data and a focus data about the image can be accommodated at the same time. A continuous image method realizes much wider views. An anti 3D image effect can be avoided by using this method. At present, an analysis of proto-type lens and experiment are being carried out. As a result, a phantom effect and a viewing area can be improved. It is possible to watch the 3D TV at any distance. Distance data are triangulated by two cameras. A plan of AVI proto type using ten thousands lenses is discussed. This method is compared with four major conventional methods. As a result, it is revealed that this method can make the efficient use of integral photography and varifocal type method. In the case of integral photography, a miniaturization of this system is possible. But it is difficult to get actual focus. In the case of varifocal type method, there is no problem with focusing, but the miniaturization is impossible. The theory investigated in this paper makes it possible to solve these problems.

We present experiments to illustrate tracking of occluded objects using three-dimensional (3D) integral imaging (II). Tracking of heavily occluded objects by conventional two-dimensional (2D) image processing may be difficult owing to the superposition of occlusion noise and object details. The effects of occlusion are remedied by 3D computational II reconstruction. We use a summation of absolute difference (SAD) algorithm between pixels of consecutive frames of a moving object for 3D tracking. SAD algorithm is not robust for 2D images of occluded objects; 3D computation reconstruction of the scene allows implementation of a SAD algorithm by reducing occlusion effects. Experimental results demonstrate 3D tracking of occluded objects. To the best of our knowledge, this is the first report on 3D tracking of objects using II.

A novel compressive 3D imaging spectrometer based on the coded aperture snapshot spectral imager (CASSI) is proposed. By inserting a microlens array (MLA) into the CASSI system, one can capture spectral data of 3D objects in a single snapshot without requiring 3D scanning. The 3D spatio-spectral sensing phenomena is modelled by computational integral imaging in tandem with compressive coded aperture spectral imaging. A set of focal stack images is reconstructed from a single compressive measurement, and presented as images focused on different depth planes where the objects are located. The proposed optical system is demonstrated with simulations and experimental results.

In order to present the visual resolution effects of different integral imaging systems in three-dimensional (3D) display applications, a method of comparing the resolutions of integral imaging 3D display based on human vision is proposed. The relative resolution parameter is defined by analyzing the relation between the integral imaging 3D system and resolving power of human eyes on the best viewing condition, and its relationship with the visual resolution effect viewed in the integral imaging 3D display system is analyzed. The results indicate that the size of lens array has an important influence on the visual resolution effect in the integral imaging 3D display system. According to the theoretical results of two examples show that the difference in relative resolution parameter between the systems is 1.75 times of that of traditional one. The experimental results are consistent with the theoretical analyses. The method proposed in this paper could have a certain directive significance for 3D display resolution evaluation.

Conventional 3D Integral imaging suffers from limited image depth range due to the fixed distance between the display panel and the lens array, while digital Fresnel holography suffers from a narrow viewing angle due to the lack of a high resolution spatial light modulator. This paper proposes an original system which combines the advantages of these two techniques to provide an integral imaging system of a reasonable viewing angle with accommodation cues.

A method for distortion-tolerant recognition of objects in water using three-dimensional (3D) integral imaging with a neural network classification architecture is presented. Recognition algorithms are developed and experimental results are presented with rotation-variable 3D objects. To test the robustness of the system, objects are placed under a variety of water conditions, including variable Maalox-induced scattering levels and occlusion using pine needles. Neural networks have long been used for two-dimensional recognition and have recently been used for 3D digital holographic recognition. To the best of our knowledge, this is the first use of neural networks for passive 3D integral imaging and recognition of underwater objects.

Applying fuzzy theory in selecting the image quality factors of 3D televisions

3D image reconstruction using light microscope modalities without exogenous contrast agents is proposed and investigated as an approach to produce 3D images of biological samples for live imaging applications. Multimodality and multispectral imaging, used in concert with this 3D optical sectioning approach is also proposed as a way to further produce contrast that could be specific to components in the sample. The methods avoid usage of contrast agents. Contrast agents, such as fluorescent or absorbing dyes, can be toxic to cells or alter cell behavior. Current modes of producing 3D image sets from a light microscope, such as 3D deconvolution algorithms and confocal microscopy generally require contrast agents. Zernike phase contrast (ZPC), transmitted light brightfield (TLB), darkfield microscopy and others can produce contrast without dyes. Some of these modalities have not previously benefitted from 3D image reconstruction algorithms, however. The 3D image reconstruction algorithm is based on an underlying physical model of scattering potential, expressed as the sample’s 3D absorption and phase quantities. The algorithm is based upon optimizing an objective function - the I-divergence - while solving for the 3D absorption and phase quantities. Unlike typical deconvolution algorithms, each microscope modality, such as ZPC or TLB, produces two output image sets instead of one. Contrast in the displayed image and 3D renderings is further enabled by treating the multispectral/multimodal data as a feature set in a mathematical formulation that uses the principal component method of statistics.

X-ray imaging offers a number of unique properties that are favorable for NDE applications, including large penetration depth, elemental specificity, and relatively low radiation damage. While direct-projection type x-ray systems with a few um resolution have been widely deployed, recent advances in x-ray optics and imaging methodology have lead to lens-based x-ray microscopes with better than 60-nm resolution, and with integrated 3D imaging and material analysis capabilities. Used independently or in combination with established techniques based on visible light and electron microscopy, these new high-resolution x-ray systems introduces many attractive new capabilities for studying structures at micrometer to tens-of-nm scale.

We demonstrate a novel method for fabricating glass microlens (arrays) with single step on conventional lens surface. In this method, the glass microlens can be achieved by only one step with sol gel glass material. The microlens aperture and focus length can be controlled easily and uniformly. The fabricated sample shows good focusing property. This work will be useful to improve the performance of compound eyes optical system such as camera, telescope, 3D integral imaging and so on.