Semi-supervised Depth Estimation from a Single Image Based on Confidence Learning

Depth estimation from a single image is very challenging due to the inherent ambiguity of mapping a color image to a depth map. Previous work tackles this problem by exploiting various levels of features with multi-scale deep convolutional neural networks. However, most of the local geometric structure related monocular depth cues are lost when being propagated through convolutional neural network. Moreover, the error of depth cues related to local geometric structures is not considered in the loss function. In this work, we propose the GeoCueDepth convolutional neural network to exploit local geometric structure cues and propose a training loss that takes the geometric error into consideration, which significantly improve the performance of depth prediction in both accuracy and sharpness. Experiments show that the proposed method achieves 0.122 average relative error and 0.078 square relative error on the NYU Depth v2 data set, which outperforms state-of-the-art monocular depth estimation approaches.

Deep neural networks have been applied to a wide range of problems in recent years. Convolutional neural network is applied to the problem of determining the depth from a single camera image (monocular depth). Eight different networks are designed to perform depth estimation, each of them suitable for a feature level. Networks with different pooling sizes determine different feature levels. After designing a set of networks, these models may be combined into a single network topology using graph optimization techniques. This "semiparallel deep neural network (SPDNN)" eliminates duplicated common network layers and can be further optimized by retraining to achieve an improved model compared to the individual topologies. Four SPDNN models are trained and have been evaluated at two stages on the KITTI dataset. The ground truth images in the first part of the experiment are provided by the benchmark, and for the second part, the ground truth images are the depth map results from applying a state-of-the-art stereo matching method. The results of this evaluation demonstrate that using postprocessing techniques to refine the target of the network increases the accuracy of depth estimation on individual mono images. The second evaluation shows that using segmentation data alongside the original data as the input can improve the depth estimation results to a point where performance is comparable with stereo depth estimation. The computational time is also discussed in this study. (C) 2018 SPIE and IS&T

Self-mixing interferometry (SMI) is superior to other laser interferometry methods due to its simplicity and compactness. However, SMI signals are often complex and difficult to process due to interference in the form of variations in the effective reflectivity of the target, noisy signals, complex signal shapes, and other dependencies. Deep neural networks have been a very popular area of research in computer artificial intelligence in recent years, allowing more implicit features to be uncovered than traditional shallow machine learning. It has been shown that convolutional and back-propagation neural networks can be used for SMI signal processing. There are also studies that have used machine learning genetic algorithms for absolute distance measurement. Based on the above, this study used convolutional neural networks to form a deep neural network for absolute distance measurement based on SMI technology. We first trained the deep convolutional neural network at different feedback strengths. The results of the Convolutional Neural Network (CNN) model showed a coefficient of determination of 0.9987. which is consistent with the required model. The trained network was then used to estimate absolute distances with and without the addition of noise. The comparison proves that the proposed method is noise-proof and has high adaptability for measurements under different conditions.

To take full advantage of the information of images captured by drones and given that most existing monocular depth estimation methods based on supervised learning require vast quantities of corresponding ground truth depth data for training, the model of unsupervised monocular depth estimation based on residual neural network of coarse–refined feature extractions for drone is therefore proposed. As a virtual camera is introduced through a deep residual convolution neural network based on coarse–refined feature extractions inspired by the principle of binocular depth estimation, the unsupervised monocular depth estimation has become an image reconstruction problem. To improve the performance of our model for monocular depth estimation, the following innovations are proposed. First, the pyramid processing for input image is proposed to build the topological relationship between the resolution of input image and the depth of input image, which can improve the sensitivity of depth information from a single image and reduce the impact of input image resolution on depth estimation. Second, the residual neural network of coarse–refined feature extractions for corresponding image reconstruction is designed to improve the accuracy of feature extraction and solve the contradiction between the calculation time and the numbers of network layers. In addition, to predict high detail output depth maps, the long skip connections between corresponding layers in the neural network of coarse feature extractions and deconvolution neural network of refined feature extractions are designed. Third, the loss of corresponding image reconstruction based on the structural similarity index (SSIM), the loss of approximate disparity smoothness and the loss of depth map are united as a novel training loss to better train our model. The experimental results show that our model has superior performance on the KITTI dataset composed by corresponding left view and right view and Make3D dataset composed by image and corresponding ground truth depth map compared to the state-of-the-art monocular depth estimation methods and basically meet the requirements for depth information of images captured by drones when our model is trained on KITTI.

Sun orientation estimation from a single image using short-cuts in DCNN

We propose a method to perform monocular relative depth perception using a passive visual sensor. Specifically, the proposed method makes depth estimation with a superpixel based regression model based on features extracted by a deep convolutional neural network. We have established and conducted an analysis of the key components required to create a high-efficiency pipeline to solve the depth estimation problem with superpixel-level regression and deep learning. The key contributions of our method compared to prior works are as follows. First, we have drastically simplified the depth estimation model while attaining near state-of-the-art prediction performance, through two important optimizations: the idea of the depth estimation model is completely based on superpixels that very effectively reduces the dimensionality; additionally, we exploited the scale invariant mean squared error loss function which incorporates a pairwise term with linear time complexity. Additionally, we have developed optimizations of the superpixel feature extraction, that leverage GPU computing to achieve real-time performance (over 50fps during training) Furthermore, this model does not perform up-sampling, which avoids many issues and difficulties that one would otherwise have to deal with. To perpetuate future research in this area we have created a synchronized multiple-view depth estimation training dataset that is available to the public.

Data driven method has been investigated in fault diagnosis. This kind of methods usually need of a large number of labeled data in order to obtain a good model. However, the unlabeled data dominates the data availability in the real industries, which impedes the successfully application of these methods. To remedy this situation, this research investigates a new semi-supervised fault diagnosis method, and it aims at training an adaptive model by using the small labeled data and large unlabeled data. Firstly, the fault signal is transformed to the RGB format. Then, Convolutional Neural Network (CNN) is trained to generate the fault features combining with the Correlation Alignment (CORAL) loss. Finally, the Transfer Component Analysis (TCA) is adopted to classify the fault features to improve the prediction accuracy. The proposed semi-supervised method is conducted on three bearing datasets. The experiments with different proportions of labeled data are done to validate the proposed CNN with CORAL and TCA. The comparisons between the proposed CNN model with other popular deep learning methods are also presented. The results show that CNN with CORAL and TCA is potential for fault diagnosis, and the prediction accuracies of proposed method outperform other comparisons methods.

Depth estimation plays a pivotal role in advancing human-robot interactions, especially in indoor environments where accurate 3D scene reconstruction is essential for tasks like navigation and object handling. Monocular depth estimation, which relies on a single RGB camera, offers a more affordable solution compared to traditional methods that use stereo cameras or LiDAR. However, despite recent progress, many monocular approaches struggle with accurately defining depth boundaries, leading to less precise reconstructions. In response to these challenges, this study introduces a novel depth estimation framework that leverages latent space features within a deep convolutional neural network to enhance the precision of monocular depth maps. The proposed model features dual encoder-decoder architecture, enabling both color-to-depth and depth-to-depth transformations. This structure allows for refined depth estimation through latent space encoding. To further improve the accuracy of depth boundaries and local features, a new loss function is introduced. This function combines latent loss with gradient loss, helping the model maintain the integrity of depth boundaries. The framework is thoroughly tested using the NYU Depth V2 dataset, where it sets a new benchmark, particularly excelling in complex indoor scenarios. The results clearly show that this approach effectively reduces depth ambiguities and blurring, making it a promising solution for applications in human-robot interaction and 3D scene reconstruction.

Deep neural networks, especially convolutional neural networks, have been successfully applied to image denoising tasks. Along with advances in network architectures, many attempts have been made to find an alternative loss function to the widely used L1‐loss and L2‐loss. However, the perception‐distortion tradeoff was recently demonstrated; thus, advanced loss functions such as adversarial loss from generative adversarial networks can only improve the perceptual image quality at the expense of distortion. This Letter shows that distortion can be further decreased when an image denoising network is trained using modified versions of ground‐truth (GT) (defined as pseudo‐ground‐truth (PGT)) images that are obtained by combining the original GT images and initially denoised images. Experimental results show that the proposed denoising network that is trained to predict both PGT and GT images produces denoised images closer to GT images.

Estimating depth information from a single image has recently attracted great attention in various vision-based applications such as mobile robot navigation. Although there are numerous depth map generation methods, little effort has been done on the depth estimation from a single indoor scene. In this paper, we propose a novel method for estimating depth from a single indoor image via nonlinear diffusion and image segmentation techniques. One important advantage of our approach is that no learning scheme is required to estimate a depth map. Based on the proposed method, we obtain visually plausible depth estimation results even with the presence of occlusions or clutters in the single indoor image. From experimental results, we confirm that the proposed algorithm provides reliable depth information under various indoor environments.

Robust depth estimation on real-world light field images using Gaussian belief propagation

Identification of crops is an important topic in the agricultural domain. Hyperspectral remote sensing data are very useful for crop feature extraction and classification. Remote sensing data is an unstructured data and Convolutional Neural Network (CNN) can work on unstructured data efficiently. This paper presents an evaluation of CNN for crop classification using the Indian Pines standard dataset obtained from the AVIRIS sensor and the study area dataset obtained from the EO-1hyperion sensor. Optimized CNN has been tuned by training the model on different parameters. It has been compared with two classification algorithms: Deep Neural Network (DNN) and Convolutional Autoencoder. According to the test results, the proposed optimized CNN model provided better results as compared to the other two methods. CNN has given 97 ± 0.58% overall accuracy for the Indian Pines standard dataset and 78 ± 2.43% for our study area dataset.

Depth estimation is an essential component in understanding the 3D geometry of a scene. In comparison to traditional depth estimation methods such as structure from motion and stereo vision matching, determining depth relation using a single camera is challenging. The recent advancements in convolutional neural networks have accelerated the research in monocular depth estimation. However, most technologies infer depth maps using lower resolution images due to network capacity and complexity issues. Another challenge in depth estimation is ambiguous and sparse depth maps. These issues are caused due to labeling errors, hardware faults, or occlusions. This paper presents a novel end-to-end trainable convolutional neural network architecture – depth transverse transformer network (DTTNet). The proposed network is designed and optimized to perform monocular depth estimation. This network aims at exploring the multi-resolution representation to perform pixel-wise depth estimation more accurately. In order to improve the accuracy of depth estimation, different kinds of ad hoc networks are proposed subsequently. Extensive computer simulations on NYU Depth V2 and SUN RGB-D dataset demonstrate the effectiveness of the proposed DTTNet against state-of-the-art methods. DTTNet can potentially optimize depth perception in intelligent systems such as automated driving and video surveillance applications, computational photography, and augmented reality. The source code is available at https://github.com/shreyaskamathkm/DTTNet

Evaluation of the benchmark datasets for testing the efficacy of deep convolutional neural networks

Gait recognition is a noncontact biometric procedure that determines the identity or health status of a person by analyzing his or her walking posture and habits, including skeletal and joint movements. The most remarkable feature of this method is the possibility of conducting recognition without demanding much cooperation from participants. Therefore, this recognition technique has attracted much attention from scholars. Additionally, because of the rapid development of graphics processing unit technology, related hardware and computation performance, the applications of deep-learning technology are considerably enhanced. The objective of this study was to apply a deep neural network (DNN), which employs deep-learning technology, to achieve gait-based automatic pedestrian detection and recognition. In contrast to using wearable devices to precisely capture skeletal and joint movements, pedestrian color-image sequences were used as input in this study. Subsequently, a pretraining convolutional neural network (CNN) was employed to capture pedestrian location and extract pedestrian dense optical flow to serve as concrete low-level feature inputs. Then, a finely-tuned DNN based on the wide residual network was employed to extract high-level abstract features. In addition, to overcome the difficulty of obtaining local temporal features by using a 2D CNN, part of the 3D convolutional structure was introduced into the CNN. This design enabled use of limited memory to acquire more effective features and enhance the DNN performance. The experimental results show that the proposed method has exceptional performance for pedestrian detection and recognition.