<title>Three-dimensional object recognition and visualization using integral imaging</title>

A method enhancing the depth discrimination of the computational integral imaging is proposed. The proposed method captures two sets of the elemental images under random pattern and regular uniform illumination. The captured two sets of real-valued elemental images are coded into one set of the complex-valued elemental images. Finally, the depth section image of the objects is reconstructed by applying computational integral imaging technique to the generated complex-valued elemental images. Experiment and simulation results show that the proposed method enhances the depth discrimination of the depth sectioning by suppressing blurred images in off-focused planes.

Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding

We present an optical method for information watermarking of three-dimensional (3D) objects by digital holography. A hidden image is embedded by double phase encoding in a phase-shift digital hologram of the 3D object. We decode the watermarked hologram to reconstruct the hidden image and the 3D object. We use either the entire hologram or a part of it to decode the hidden image. Experiments are presented to illustrate the ability to recover both the 3D object and the decoded hidden image. Digital holograms of the 3D object are obtained by optical experiments. The watermarking process, 3D object reconstruction, and hidden image recovery are performed digitally. To the best of our knowledge, this is the first report of 3D object watermarking by use of a phase encoding technique and digital holography.

We propose a framework for three-dimensional (3D) object recognition and classification in very low illumination environments using convolutional neural networks (CNNs). 3D images are reconstructed using 3D integral imaging (InIm) with conventional visible spectrum image sensors. After imaging the low light scene using 3D InIm, the 3D reconstructed image has a higher signal-to-noise ratio than a single 2D image, which is a result of 3D InIm being optimal in the maximum likelihood sense for read-noise dominant images. Once 3D reconstruction has been performed, the 3D image is denoised and regions of interest are extracted to detect 3D objects in a scene. The extracted regions are then inputted into a CNN, which was trained under low illumination conditions using 3D InIm reconstructed images, to perform object recognition. To the best of our knowledge, this is the first report of utilizing 3D InIm and convolutional neural networks for 3D training and 3D object classification under very low illumination conditions.

Light detection and ranging (LiDAR) is widely used in the automotive industry as it can provide point cloud information containing precise distances. Three-dimensional (3D) object detection based on LiDAR point clouds is significant for environment perception tasks. However, feature learning for point clouds remains challenging. This paper proposes a two-stage voxel-based LiDAR 3D object detector, referred to as density-aware and neighborhood attention (DenNet), that focuses on the neighborhood information of objects. DenNet mainly integrates two modules: voxel density-aware (VDA) and neighborhood attention (NA). VDA introduces density information of the point cloud. Here, point cloud density information was added as voxel features in the voxel-based framework to alleviate the information loss during voxelization. Additionally, to extract neighbor information, the characteristics of 3D objects were analyzed for traffic scenes. The NA mechanism was adopted, which localizes the receptive field for each query to its nearest neighboring points. DenNet yielded competitive results, as compared with state-of-the-art methods, for the KITTI and One Million Scenes (ONCE) datasets; notably, it afforded an improvement of 3.96% relative to the baseline mean average precision on the more challenging ONCE dataset.

The biggest advantage of time-of-flight secondary ion mass spectrometry (TOF-SIMS) is imaging performance. TOF-SIMS is the only surface analysis technique which can provide the elemental and molecular ion images with high spatial resolution and high sensitivity. However, acquiring accurate images of uniform intensity from three-dimensional (3D) objects having curved surfaces is challenging because the ion extraction field in a TOF-SIMS system is ideally suited for the measurement of flat samples. When the authors observe the samples which have curved surfaces, the observable region is often restricted; it is, therefore, impossible to discuss the chemical abundance of components at various locations on the sample surface. Several methods to solve this problem have been proposed, e.g., delayed extraction or embedding the sample, but each has some disadvantages. Therefore, the authors have developed a curved surface sample holder for the uniform extraction of ions from 3D objects. This sample holder succeeded to expand the observable region dramatically without any significant drawbacks. In this article, the capability for uniform imaging afforded by the newly designed sample holder will be demonstrated.

Three-Dimensional (3D) object recognition has been receiving more and more attention in the field of Computer Vision, as the development of AI algorithms and advancements of hardware components allow for better results nowadays more than ever before. Neural Networks are used to better classify different types of objects, these are being fed with pre-processed 3D objects represented in a 2D domain for better highlighting the important features and for a better performance. In this paper, we propose a strategy of feeding 2D representations of 3D meshes made by using different types of slicing to a CNN. In this way, we want to minimize the information loss and the redundancy the data suffers going from a 3D representation to a 2D one. To assess the results, we used Princeton's state-of-the-art ModelNet10 dataset, consisting of 4899 3D objects divided into 10 categories.

Holographic imaging offers a reliable and fast method to capture the complete three-dimensional (3D) information of the scene from a single perspective. We review our recently proposed single-channel optical system for generating digital Fresnel holograms of 3D real-existing objects illuminated by incoherent light. In this motionless holographic technique, light is reflected, or emitted from a 3D object, propagates through a spatial light modulator (SLM), and is recorded by a digital camera. The SLM is used as a beamsplitter of the single-channel incoherent interferometer, such that each spherical beam originated from each object point is split into two spherical beams with two different curve radii. Incoherent sum of the entire interferences between all the couples of spherical beams creates the Fresnel hologram of the observed 3D object. When this hologram is reconstructed in the computer, the 3D properties of the object are revealed.

Recently, a method for synthesizing a hologram of three dimensional (3D) objects from captured light field array is demonstrated. The 3D objects can be captured under incoherent light illumination using a micro lens array and their orthographic projection view images are generated from the captured elemental images. The synthesized orthographic projection view images are then multiplied by the corresponding phase functions and combined to form a digital hologram. For the first time, we analysis the performance of synthesized hologram under photon counting (low light imaging) conditions. The feasibility of this technique is experimentally verified by recording the orthographic projection images using a micro lens array and the reconstructed photon counted hologram is presented with varying photoncounting measurements.

A novel method for three-dimensional (3-D) object rotation-invariant recognition by use of perspective array and the Synthetic Discriminant Function (SDF) is proposed. With high relevancy of perspective array, the conversion of depth information of 3-D object to 2-D perspective images by use of microlens array makes 3-D object recognition is possible accomplished through the technology of 2-D image. The SDF is synthesized by use of the computer, and 3-D object rotation-invariant recognition is accomplished with the aberration-invariant characteristic of SDF. Correlative analysis has been done for the conversion and recognition process, the SDF has been synthesized by use of Matlab process, and rotation-invariant recognition of dice, a 3-D object, has been accomplished by use of the method of match filtering. The result showed that the proposed method is performed better.

Recognizing 3D objects in the presence of noise, varying mesh resolution, occlusion and clutter is a very challenging task. This paper presents a novel method named Rotational Projection Statistics (RoPS). It has three major modules: local reference frame (LRF) definition, RoPS feature description and 3D object recognition. We propose a novel technique to define the LRF by calculating the scatter matrix of all points lying on the local surface. RoPS feature descriptors are obtained by rotationally projecting the neighboring points of a feature point onto 2D planes and calculating a set of statistics (including low-order central moments and entropy) of the distribution of these projected points. Using the proposed LRF and RoPS descriptor, we present a hierarchical 3D object recognition algorithm. The performance of the proposed LRF, RoPS descriptor and object recognition algorithm was rigorously tested on a number of popular and publicly available datasets. Our proposed techniques exhibited superior performance compared to existing techniques. We also showed that our method is robust with respect to noise and varying mesh resolution. Our RoPS based algorithm achieved recognition rates of 100, 98.9, 95.4 and 96.0 % respectively when tested on the Bologna, UWA, Queen’s and Ca’ Foscari Venezia Datasets.

Three-dimensional object recognition using joint fractional Fourier transform correlators with the help of digital Fresnel holography

We report the generation of a real-time large computer generated hologram (CGH) using the wavefront recording plane (WRP) method with the aid of a graphics processing unit (GPU). The WRP method consists of two steps: the first step calculates a complex amplitude on a WRP that is placed between a 3D object and a CGH, from a three-dimensional (3D) object. The second step obtains a CGH by calculating diffraction from the WRP to the CGH. The disadvantages of the previous WRP method include the inability to record a large three-dimensional object that exceeds the size of the CGH, and the difficulty in implementing to all the steps on a GPU. We improved the WRP method using Shifted-Fresnel diffraction to solve the former problem, and all the steps could be implemented on a GPU. We show optical reconstructions from a 1,980 × 1,080 phase only CGH generated by about 3 × 10(4) object points over 90 frames per second. In other words, the improved method obtained a large CGH with about 6 mega pixels (1,980 × 1,080 × 3) from the object points at the video rate.

In this paper we propose an improved version of the computational integral imaging reconstruction (CIIR) for visualizing a partially occluded object by utilizing an image inpainting technique. In the proposed method the elemental images for a partially occluded three-dimensional (3D) object are recorded through the integral imaging pickup process. Next, the depth of occlusion within the elemental images is estimated using two different CIIR methods, and the weight mask pattern for occlusion is generated. After that, we apply our image inpainting technique to the recorded elemental images to fill in the occluding area with reliable data, using information from neighboring pixels. Finally, the inpainted elemental images for the occluded region are reconstructed using the CIIR process. To verify the validity of the proposed system, we carry out preliminary experiments in which faces are the objects. The experimental results reveal that the proposed system can dramatically improve the quality of a reconstructed CIIR image.

Optical 3D watermark based digital image watermarking for telemedicine