Deep Learning-Based Multi-class 3D Objects Classification Using Digital Holographic Complex Images

We propose a framework for three-dimensional (3D) object recognition and classification in very low illumination environments using convolutional neural networks (CNNs). 3D images are reconstructed using 3D integral imaging (InIm) with conventional visible spectrum image sensors. After imaging the low light scene using 3D InIm, the 3D reconstructed image has a higher signal-to-noise ratio than a single 2D image, which is a result of 3D InIm being optimal in the maximum likelihood sense for read-noise dominant images. Once 3D reconstruction has been performed, the 3D image is denoised and regions of interest are extracted to detect 3D objects in a scene. The extracted regions are then inputted into a CNN, which was trained under low illumination conditions using 3D InIm reconstructed images, to perform object recognition. To the best of our knowledge, this is the first report of utilizing 3D InIm and convolutional neural networks for 3D training and 3D object classification under very low illumination conditions.

We present an optical method for information watermarking of three-dimensional (3D) objects by digital holography. A hidden image is embedded by double phase encoding in a phase-shift digital hologram of the 3D object. We decode the watermarked hologram to reconstruct the hidden image and the 3D object. We use either the entire hologram or a part of it to decode the hidden image. Experiments are presented to illustrate the ability to recover both the 3D object and the decoded hidden image. Digital holograms of the 3D object are obtained by optical experiments. The watermarking process, 3D object reconstruction, and hidden image recovery are performed digitally. To the best of our knowledge, this is the first report of 3D object watermarking by use of a phase encoding technique and digital holography.

This paper proposes a method for recognition and classification of 3D objects using 2D moments and HMLP network. The 2D moments are calculated based on 2D intensity images taken from multiple cameras that have been arranged using multiple views technique. 2D moments are commonly used for 2D pattern recognition. However, the current study proves that with some adaptation to multiple views technique, 2D moments are sufficient to model 3D objects. In addition, the simplicity of 2D moment's calculation reduces the processing time for feature extraction, thus decreases the recognition time. The 2D moments were then fed into a neural network for classification of the 3D objects. In the current study, hybrid multi-layered perceptron (HMLP) network is proposed to perform the classification. Two distinct groups of objects that are polyhedral and free-form objects were used to access the performance of the proposed method. The recognition results show that the proposed method has successfully classified the 3D object with the accuracy of up to 100%.

In this paper, we present a deep CNN-based approach for multi-class classification of three-dimensional (3-D) objects using phase-only digital holographic information. The 3-D objects considered for the multi-class (four-class) classification task are ‘triangle-square’, ‘circle-square’, ‘square-triangle’, and ‘triangle-circle’. The 3-D object ‘triangle-square’ is considered for Class-1 and the remaining 3-D objects ‘circle-square’, ‘square-circle’, and ‘triangle-circle’ are considered for Class-2, Class-3, and Class-4. The digital holograms of 3-D objects were created using the two-step Phase-Shifting Digital Holography (PSDH) technique and were computationally post-processed to obtain phase-only digital holographic data. Subsequently, a deep CNN was trained on a phase-only image dataset consisting of 2880 images to produce the results. The loss and accuracy curves are presented to validate the performance of the model. Additionally, the results are validated using metrics such as the confusion matrix, classification report, Receiver Operating Characteristic (ROC) curve, and precision-recall curve.

Digital holography is a powerful tool for noncontact quantitative phase imaging. According to the relative incident angle between the object beam and the reference beam, digital holography is grouped into on-axis and off-axis digital holography, The measurable area is narrow in off-axis digital holography, on-axis digital holography suffers from image blurring. Phase-shifting technique is usually used to obtain the high-quality object image. However, the phase shifting technique requires to record multiple phase-shifted holograms. The most conventional holography configuration requires a separately generated reference and object beams that result in a low stability. The paper presents an One-shot common-path phase-shifting holography based on micro polarizer camera and large-shearing Wollaston Prism. The system employs a commercial micro polarizer camera and a doubly-refractive prism with large shearing. The Wollaston prism separates the incoming beam into two orthogonally polarized components ,brings the reference and object from the two-windows to overlap at the lateral shearing region. The two light beams transmit through the quarter wave plate(QWP) and pixelated micro polarizer array(PMA) camera, QWP is used to transform the orthogonally polarized light into orthogonal circular components, The circular polarizations interfere at CCD after passing through the micro polarizer array. The data captured by PMA camera can be parsed into four phase shifting fringe images corresponding to each direction of the four polarizations. The interpolation method is used to obtain the same resolution as the original image. The phase distribution of the specimen can be retrieved using the four-step algorithm. Finally, experiments are conducted to prove the validity of the proposed method. The results demonstrate the capability and applicability of the system .

Three-dimensional object recognition using joint fractional Fourier transform correlators with the help of digital Fresnel holography

This paper proposes a quality assessment of focusing criteria for imaging in digital off-axis holography. In literature, several refocus criteria have been proposed in the past to get the best refocus distance in digital holography. As a general rule, the best focusing plane is determined by the reconstruction distance for which the criterion function presents a maximum or a minimum. To evaluate the robustness of these criteria, a set of thirteen criteria is compared with application on both amplitude and phase images from off-axis holographic data. Experimental results lead to define general rule and to exhibit the most robust criteria for accurate and rapid refocusing in digital holography.

We propose a temporal multiplexing method of two holograms based on off-axis digital holography using acousto-optic modulators. The proposed method can overcome the limitation of the frame rate of an image sensor for digital holography.

Accurately determining the focus plane of the reconstructed image is crucial for obtaining high-quality reconstructed images in the process of digital hologram reconstruction. In this paper, a focusing evaluation function based on S-th power weighted neighborhood correlation coefficient (SPWNCC) is proposed to realize automatic focusing of the reconstructed image in off-axis digital Fresnel holography. The Fresnel transform method is utilized as the off-axis digital holographic reconstruction algorithm. Both the numerical simulation and optical experiment results are given to verify the validity of the proposed autofocusing method. The obtained focusing curve can maintain good unimodality and noise immunity performance over a large search range (approximately 800 mm). The proposed SPWNCC based focusing evaluation function has a certain guiding significance on the automatic focusing of off axis digital holographic reconstructed images of long distance recorded objects.

Currently, lung cancer has one of the highest mortality rates because it is often caught too late. Therefore, early detection is essential to reduce the risk of death. Pulmonary nodules are considered key indicators of primary lung cancer. Developing an efficient and accurate computer-aided diagnosis system for pulmonary nodule detection is an important goal. Typically, a computer-aided diagnosis system for pulmonary nodule detection consists of two parts: candidate nodule extraction and false-positive reduction of candidate nodules. The reduction of false positives (FPs) of candidate nodules remains an important challenge due to morphological characteristics of nodule height changes and similar characteristics to other organs. In this study, we propose a novel multi-scale heterogeneous three-dimensional (3D) convolutional neural network (MSH-CNN) based on chest computed tomography (CT) images. There are three main strategies of the design: (1) using multi-scale 3D nodule blocks with different levels of contextual information as inputs; (2) using two different branches of 3D CNN to extract the expression features; (3) using a set of weights which are determined by back propagation to fuse the expression features produced by step 2. In order to test the performance of the algorithm, we trained and tested on the Lung Nodule Analysis 2016 (LUNA16) dataset, achieving an average competitive performance metric (CPM) score of 0.874 and a sensitivity of 91.7% at two FPs/scan. Moreover, our framework is universal and can be easily extended to other candidate false-positive reduction tasks in 3D object detection, as well as 3D object classification.

This paper proposes an effective method for recognition and classification of 3D objects using multiple views technique and neural networks system. In the processing stage, we propose to use 2D moment invariants as the features for modeling 3D objects. 2D moments have been commonly used for 2D object recognition. However, we have proved that with some adaptation to multiple views technique, 2D moments are sufficient to model 3D objects. In addition, the simplicity of 2D moments calculation reduces the processing time for feature extraction, hence increases the system efficiency. In the recognition stage, we propose a cascaded multilayered perceptron (c-MLP) network for matching and classification. The c-MLP contains two MLP networks which are arranged in a serial combination. This proposed method has been tested using two groups of object, polyhedral and free-form objects. We also compare our method with standard MLP network. Our results show that the proposed method can successfully be applied to 3D object recognition. In addition, the proposed network also achieved better performance and faster convergence rate compared to the than standard MLP

Nowadays, point clouds are frequently gathered by 3D scanners such as Lidar and Kinect, which produces thousands of point cloud models. Point cloud processing is vital to 3D vision, especially 3D object recognition, positioning, and navigation technology. Addressing uneven data density caused by coordinate frame transformations and the inherent problem of insufficient context connection in point clouds, the DiM-PCNet (Multi-scale and Multi-level Point Clouds Classification Network, and the Di is a prefix to represent double M in Multi-scale and Multi-level) is proposed in this paper. DiM-PCNet is provided for object classification with multi-scale and multi-level features. We encode the point cloud in multi-scale and fully fuse the features with the raw point cloud for keeping the context relationship. In DiM-PCNet, we sample the point clouds from eight parts for multi-scales feature extraction. The multi-scales features are fully fused by multi-level pyramid models. The multi-scale and multi-level strategies are applied in DiM-PCNet, in which the abundant and important features of point clouds are extracted and utilized in the 3D object classification. It is worth noting that the DiM-PCNet feature block can be embedded into the segmentation net, where the accuracy achieved is 87.1%. We conducted experiments on ShapeNet and ModelNet40 and the experimental results show that DiM-PCNet achieves state-of-the-art performance in 3D object classification. The experiment shows competitive performance on robustness and segmentation tasks.

We describe a numerical space-shifting reconstruction approach in digital holography. This method is able to remove the dc term in the reconstruction very effectively by utilizing the periodicity and the space-shifting property of inverse discrete Fourier transform. Since the entire process does not need any additional holograms and specific requirements to recording optics, this approach can be a really convenient, practical, and widely effective way to remove the dc term from in-line or off-axis digital holography.

Multi-wavelength digital holography is a very powerful approach for surface shape measurements. It has the advantage of being contact-less, non-intrusive, and yields full-field surface shape data without any requirement for scanning. When dealing with off-axis digital holography and spatial multiplexing of two-wavelength digital holograms, the method becomes real-time, in the sense that the surface shape can be measured at each time instant at which the holograms are recorded. Thus, phase shifting and sequential recording are suppressed. However, due to the roughness of the inspected surface, speckle decorrelation occurs and noise is included in the final data. The noise amount in the data must be investigated in order to define the best processing approach for holograms. This paper proposes the analysis of the standard deviation of noise in surface-shape data from two-wavelength spatially-multiplexed digital holograms. The influence of noise on the measurements of the surface shape is described by an analytical approach. Relationships to quantify the minimum measurable surface height is given by taking into account the experimental parameters of the set-up. These parameters are related to the spatial bandwidths, modulation of holograms, saturation ratio, number of electrons in pixels, readout noise, quantization noise, photon noise, and speckle decorrelation due to roughness. The theoretical modeling is discussed and analysis when considering practical situation for industrial surface shape measurements.

A cross-entropy method for the classification of 3D objects with curved surfaces is proposed. In the proposed approach, small cross-entropy between the shape histograms of two objects indicates that the two objects are similar, where the shape histogram is defined as the distribution of the shape indices computed at pixels on the object. Experiments with real range images show that the cross-entropy method can be used to classify 3D objects with curved surfaces.