Hierarchical upsampling for fast image-based depth estimation

In this study, we present a method for all-around depth estimation from multiple omnidirectional images for indoor environments. In particular, we focus on plane-sweeping stereo as the method for depth estimation from the images. We propose a new icosahedron-based representation and ConvNets for omnidirectional images, which we name "CrownConv" because the representation resembles a crown made of origami. CrownConv can be applied to both fisheye images and equirectangular images to extract features. Furthermore, we propose icosahedron-based spherical sweeping for generating the cost volume on an icosahedron from the extracted features. The cost volume is regularized using the three-dimensional CrownConv, and the final depth is obtained by depth regression from the cost volume. Our proposed method is robust to camera alignments by using the extrinsic camera parameters; therefore, it can achieve precise depth estimation even when the camera alignment differs from that in the training dataset. We evaluate the proposed model on synthetic datasets and demonstrate its effectiveness. As our proposed method is computationally efficient, the depth is estimated from four fisheye images in less than a second using a laptop with a GPU. Therefore, it is suitable for real-world robotics applications. Our source code is available at https://github.com/matsuren/crownconv360depth.

Self-supervised depth estimation methods can achieve competitive performance using only unlabeled monocular videos, but they suffer from the uncertainty of jointly learning depth and pose without any ground truths of both tasks. Supervised framework provides robust and superior performance but is limited by the scope of the labeled data. In this paper, we introduce SENSE, a novel learning paradigm for self-supervised monocular depth estimation that progressively evolves the prediction result using supervised learning, but without requiring labeled data. The key contribution of our approach stems from the novel use of the pseudo labels - the noisy depth estimation from the self-supervised methods. We surprisingly find that a fully supervised depth estimation network trained using the pseudo labels can produce even better results than its "ground truth". To push the envelope further, we then evolve the self-supervised backbone by replacing its depth estimation branch with that fully supervised network. Based on this idea, we devise a comprehensive training pipeline that alternatively enhances the two key branches (depth and pose estimation) of the self-supervised backbone network. Our proposed approach can effectively ease the difficulty of multi-task training in self-supervised depth estimation. Experimental results have shown that our proposed approach achieves state-of-the-art results on the KITTI dataset.

To improve the navigation safety of inland river ships and enrich the methods of environmental perception, this paper studies the recognition and depth estimation of inland river ships based on binocular stereo vision (BSV). In the stage of ship recognition, considering the computational pressure brought by the huge network parameters of the classic YOLOv4 model, the MobileNetV1 network was proposed as the feature extraction module of the YOLOv4 model. The results indicate that the mAP value of the MobileNetV1-YOLOv4 model reaches 89.25%, the weight size of the backbone network was only 47.6 M, which greatly reduced the amount of computation while ensuring the recognition accuracy. In the stage of depth estimation, this paper proposes a feature point detection and matching algorithm based on the ORB algorithm at sub-pixel level, that is, firstly, the FSRCNN algorithm was used to perform super-resolution reconstruction of the original image, to further increase the density of image feature points and detection accuracy, which was more conducive to the calculation of the image parallax value. The ships’ depth estimation results indicate that when the distance to the target is about 300 m, the depth estimation error is less than 3%, which meets the depth estimation needs of inland ships. The ship target recognition and depth estimation technology based on BSV proposed in this paper makes up for the shortcomings of the existing environmental perception methods, improves the navigation safety of ships to a certain extent, and greatly promotes the development of intelligent ships in the future.

Plausible depth prediction from a single image is a challenging task in computer vision. This paper proposes a non-parametric learning-based single image depth prediction framework in the Fourier domain. Specifically, the candidate depth maps are retrieved from a large RGBD (RGB image+Depth) dataset and transferred to the input image based on the Coarse-to-fine PatchMatch (CPM) algorithm. Then, a Fourier transform-based depth fusion strategy is designed, in which the Discrete Fourier Transform (DFT) coefficients of the warped candidate depth maps are extracted and integrated via confidence analysis. Reliable depth map of the input image can be reconstructed based on the fused DFT vectors by using the inverse Fourier transform. Experimental results demonstrate that the proposed algorithm outperforms other depth estimation methods, and is very effective for inferring convincing depth maps.

To improve the depth estimation precision at the object edges, a novel depth estimation method is proposed in this paper. In the proposed method, based on the depth characteristics of the object edges, the pixels in the image are firstly classified into three categories, and then the corresponding smoothness cost functions are derived. With the modified smoothness cost functions, the depth value at the object edges with high quality can be obtained using the proposed depth estimation method. Compared with the existing depth estimation methods for the free-viewpoint television (FTV) system, the experimental results have shown that the proposed method can greatly improve the depth estimation precision for the object edges and guarantee the correctness of the edges in the synthesized virtual view at the receiver of the FTV system.

Source depth estimation is always a problem in underwater acoustic area, because depth estimation is a nonlinear problem. Traditional depth estimation methods use a vertical line array, which has disadvantage of poor mobility due to the size of sensor array. In order to estimate source depth with a horizontal line array, we propose a matched-mode depth estimation method based on autoregressive (AR) wavenumber estimation for a moving source in shallow water waveguides. First, we estimate the mode wavenumbers using the improved AR modal wavenumber spectrum. Second, according to the mode wavenumber estimation, we estimate the mode amplitudes by the wavenumber spectrum, which is obtained by generalized Hankel transform. Finally, we estimate source depth estimation by the peak of source depth function wherein the data mode best matches the replica mode that is calculated using a propagation model. Compared with synthetic aperture beamforming, the proposed method exhibits a better performance in source depth estimation under low signal-to-noise ratio or the small range span. The robustness of the proposed method is illustrated by simulating the performance in mismatched environment.

Predicting depth maps from monocular images has made an impressive performance in the past years. However, most depth estimation methods are trained with paired image-depth map data or multi-view images (e.g., stereo pair and monocular sequence), which suffer from expensive annotation costs and poor transferability. Although unsupervised domain adaptation methods are introduced to mitigate the reliance on annotated data, rare works focus on the unsupervised cross-scenario indoor monocular depth estimation. In this article, we propose to study the generalization of depth estimation models across different indoor scenarios in an adversarial-based domain adaptation paradigm. Concretely, a domain discriminator is designed for discriminating the representation from source and target domains, while the feature extractor aims to confuse the domain discriminator by capturing domain-invariant features. Further, we reconstruct depth maps from latent representations with the supervision of labeled source data. As a result, the feature extractor learned features possess the merit of both domain-invariant and low source risk, and the depth estimator can deal with the domain shift between source and target domains. We conduct the cross-scenario and cross-dataset experiments on the ScanNet and NYU-Depth-v2 datasets to verify the effectiveness of our method and achieve impressive performance.

In this paper we present a novel method for depth estimation from stereoscopic 360-degree video. As for now, no such methods have been presented in the literature. First we show mathematical foundations for correspondence matching polar coordinates. Then, the attained formulas are simplified to show relationship to depth estimation in rectified pairs of rectangular images. The proposed method has been implemented in Depth Estimation Reference Software (DERS) developed by Motion Picture Experts Group (MPEG) of International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC). Finally, we present experimental results in form of depth maps estimated for “Dancing 360” stereoscopic omnidirectional sequence.

Assessment of depth to magnetic sources using high resolution aeromagnetic data of some parts of the Lower Benue Trough and adjoining areas, Southeast Nigeria

Translation of fluorescence imaging using molecularly targeted imaging agents for real-time assessment of surgical margins in the operating room requires a fast and reliable method to predict tumor depth from planar optical imaging. Here, we developed a dual-wavelength fluorescent molecular probe with distinct visible and near-infrared excitation and emission spectra for depth estimation in mice and a method to predict the optical properties of the imaging medium such that the technique is applicable to a range of medium types. Imaging was conducted at two wavelengths in a simulated blood vessel and an in vivo tumor model. Although the depth estimation method was insensitive to changes in the molecular probe concentration, it was responsive to the optical parameters of the medium. Results of the intra-tumor fluorescent probe injection showed that the average measured tumor sub-surface depths were 1.31 ± 0.442 mm, 1.07 ± 0.187 mm, and 1.42 ± 0.182 mm, and the average estimated sub-surface depths were 0.97 ± 0.308 mm, 1.11 ± 0.428 mm, 1.21 ± 0.492 mm, respectively. Intravenous injection of the molecular probe allowed for selective tumor accumulation, with measured tumor sub-surface depths of 1.28 ± 0.168 mm, and 1.50 ± 0.394 mm, and the estimated depths were 1.46 ± 0.314 mm, and 1.60 ± 0.409 mm, respectively. Expansion of our technique by using material optical properties and mouse skin optical parameters to estimate the sub-surface depth of a tumor demonstrated an agreement between measured and estimated depth within 0.38 mm and 0.63 mm for intra-tumor and intravenous dye injections, respectively. Our results demonstrate the feasibility of dual-wavelength imaging for determining the depth of blood vessels and characterizing the sub-surface depth of tumors in vivo.

Depth estimation serves as the foundation for many computer vision applications. As an emerging perception sensor inspired by biological retinas, event cameras have garnered significant attention in recent years due to their high dynamic range, ultra-high temporal resolution, low latency, and other distinctive features.This paper provides a systematic review of depth estimation methods based on event cameras, including related datasets, evaluation metrics, and future development prospects. In terms of methodological classification, the approaches are divided into frame-based methods and learning-based methods (the latter further subdivided into Spiking Neural Networks (SNN) and Artificial Neural Network (ANN)) based on sensor type. Additionally, the supervision modes (supervised/unsupervised) and open-source availability of these methods are examined in detail, summarizing the algorithmic principles, performance, and contributions of various approaches. Subsequently, commonly used datasets and measurement techniques in event camera-based depth estimation are systematically organized. Finally, the development prospects of this field are discussed. We hope this paper can offer a comprehensive survey and serve as an introductory reference for researchers new to this field.

Research on binocular stereo vision phenotyping measurement for leafy vegetable based on 3DGS supervision

A method of range and depth estimation was studied using a single hydrophone based on the dispersive characteristic and time-frequency analysis for low frequency underwater acoustic pulse signals in shallow water environment. First, the signal received on a single hydrophone can be decomposed into a series of modes within the frame work of normal mode theory, and then the dispersive characteristic of the propagating modes can be analyzed using the time-frequency analysis. In order to improve the time-frequency resolution, the use of the time-frequency distribution with adaptive radial-Gaussian kernel extracts the arrival time difference of propagating modes in dispersion curve, which can be used to estimate source range. Mode energy can be extracted using binary time-frequency mask filtering based on multi-mode joint matching processing; and the source depth can be estimated by comparing the differences of the mode energy of the real data and simulated replica data, yielding a contrast function. Simulation results from a shallow-water Pekeris waveguide show that the time-frequency distribution with adaptive radial-Gaussian kernel represents well the dispersion characteristics of the underwater acoustic pulse signals, provides higher time-frequency resolution and overcomes the problem of the inherent limit for the time resolution and frequency resolution in the traditional short-time Fourier transform, so that the modes can be separated and identified more easily in the time-frequency plane. From the result of the range estimation, the different mode combinations have different results of the range estimation. The range estimation result can be obtained accurately by using the mode with high energy in the time-frequency plane. The relative error in range estimation is less than 2% by using the mode with high energy. In terms of the depth estimation, the more the number of joint matching mode, the more sharp peak and low fake peaks the contrast function has, so that the depth estimation is further improved by incorporating more modes. This research has great significance for studying the extraction and separation of low frequency underwater acoustic pulse signals.

Underground mapping is of paramount importance at archaeological sites with natural or man‐made caves. Techniques of different complexity are available for underground surveying, from compass and tape to light detection and ranging (LiDAR). However, there are scenarios where it is impossible to use heavy and/or delicate instrumentation, or with some of the more advanced techniques, long‐distance and time‐consuming fieldwork would be required. This is the case of the study of the height of water at key points inside the aquifer located in the subsoil of the Roman city of Colonia Clunia Sulpicia to assess its relation with the evolution of the city. Although subsurface radiolocation, a technique originally intended for cave mapping, seems promising for this application, its accuracy in depth estimation is not sufficient for this purpose. Therefore, the main objectives of this work are to improve the accuracy of depth estimation using radiolocation, to obtain sound data to study changes in water supply in the early centuries of the city and to hypothesize possible causes and probable consequences. Then, this work analyses the sources of error affecting the radiolocation process and develops a new method experimentally validated to improve the depth estimation accuracy. As a result, the depth of key points at Clunia has been accurately measured by radiolocation, where LiDAR or direct measurements from the water level are not possible. Finally, based on the resultant data, a chronology of the city in relation to the aquifer, and vice versa, is outlined. The new radiolocation and calculation procedure proposed in this paper is an improved subsurface location technique that can lead to a significant innovation in archaeological prospection.

While some image textures can be changed with scale, others cannot. We focus on a multi-scale features of determing the sensitivity of the texture intensity to change. This paper presents a new method of texture structure classification and depth estimation using multi-scale features extracted from a higher order of the local autocorrelation functions. Multi-scale features consist of the meansand variances of distributions, which are extracted from theautocorrelation feature vectors according to multi-level scale. In order to reduce dimensional feature vectors, we employ the Principal Component Analysis (PCA) in the autocorrelation feature space. Each training image texture makes its own multi-scale model in a reduced PCA feature space, and the test of the texture image is projected in the homogeneous PCA space of the training data. The experimental results show that the proposed multi-scale feature can be utilized notonly for texture classification, but also depth estimation in two dimensional images with texture gradients.